Bug Reporting




Introduction

If you find a problem or a bug in the WebPA tool we would really like to know about it. Once the project knows about the bug then we can fix it, make a patch available and then incorporate the fix in the next release of the WebPA tool.

Anybody can report a bug, you don’t need to be a programmer/developer. If you use the projects source forge, you don’t even need to sign in to a system or tell us who you are. All we ask is that you try and report a bug so that we can re-create the problem and understand how you found it.

What is not helpful is when people report bugs in a bad way. A bad bug report; says nothing, makes no sense, doesn’t give enough information, or gives the wrong information.

It isn’t only non-programmers who produce bad bug reports. Some of the worst bug reports come from programmers, and even from good programmers.

By giving helpful information we can determine if the problems are user error, the fault of somebody elses program or a problem with the tool we missed. We will also be able to deal with the report in a quick and timely way.

In the following sections we will hopefully explain what makes a good bug report and how you can write one.


What is the aim of a bug report…

In a nutshell, the aim of a bug report is to enable the programmer to see the program failing in front of them. You can either show them in person, or give them careful and detailed instructions on how to make it fail. If they can make it fail, they will try to gather extra information until they know the cause. If they can’t make it fail, they may have to ask you to gather that information for them.

In bug reports, try to make very clear what are actual facts (“I was at the computer and this happened”) and what are speculations (“I think the problem might be this”). Leave out speculations if you want to, but don’t leave out facts.

When you report a bug, you are doing so because you want the bug fixed. There is no point in being deliberately unhelpful: it may be their our and your problem, and you might be right to be angry with us, but the bug will get fixed faster if you help us by supplying all the information we need.

Remember WebPA is open source, then the contributors are supplying their time for free, so if too many people are rude to them then they may stop helping.



“It doesn’t work.”

Just because the WebPA tool is not working for you, doesn’t mean that the tool is broken and contains lots of bugs. If it really didn’t work at all then we would not have released the tool. Therefore, either you are doing something different, or your environment is different. By including information about the environment you are using then dealing with your problem is easier, and you may get a faster report. More information is almost always better than less.

WebPA provides a list of known bug through our source forge area. It is worth having a look through the list of bugs that have already been found. If the bug you are experiencing has already been identified, then it is probably not worth reporting it again. But if you think you have more information to add then you may want to contact the project team, or add comments to the existing bug report.

If you are not reporting a bug, but are asking for help then you need to look in all the other sources first, such as the website Support section and the JISC mail list. If you have exhausted all other avenues and this is your last resort, you need to state where you have already looked for the answer to your question. This will help us to improve our documentation and make it easier to use.



“Show me.”

One of the very best ways you can report a bug is by showing it to the programmer. However, in most cases it is not possible to stand them in front of your computer. The only possible way of taking the programmer through your problem is to write it down as if you are talking some one through your actions.

If you need to report a bug (we can’t be there for every one, everywhere), the aim of the exercise is to enable us to reproduce the problem. We want to run WebPA, do the same things as you did, and make it fail in the same way. When we are able to see the problem then we can make a fix and release a patch.

So tell us exactly what you did. If it’s a graphical program, tell us which buttons you pressed and what order you pressed them in. Again we come back to writing down your actions as if you where talking some one through what you did to produce the error. But you also need to include what the computer did, or output to the screen.



“Works for me. So what goes wrong?”

If you send a long list of inputs and actions, and when we fire up our own copy of the program and nothing goes wrong, then you may not have given us enough information.

It maybe that the problems doesn’t occur on every computer. Possibly you have misunderstood what the WebPA tool is supposed to do, and we are both looking at exactly the same display but you think it’s wrong and we are sure it’s right.

So also describe what happened. Tell us exactly what you saw. Tell us why you think what you saw is wrong; better still, tell us them exactly what you expected to see. If you say “and then it went wrong”, you have left out some very important information.

If you saw error messages then relay, carefully and precisely, what it was and where. They are important!

At this stage, the we are not trying to fix the problem: just trying to find it. We need to know what has gone wrong, and those error messages are the computer’s best effort to tell you that. Write the errors down if you have no other easy way to remember them, but it’s not worth reporting that WebPA generated an error unless you can also report what the error message was.

At this stage, we are effectively doing detective work. We don’t know what’s happened, and can’t get close enough to watch it happen, hence we have to search for clues that might give it away. Error messages, incomprehensible strings of numbers, and even unexplained delays are all just as important as fingerprints at the scene of a crime.



“So then I tried . . .”

There are a lot of things you might do when an error or bug comes up. Many of them can make the problem worse.

When something goes wrong, immediately stop doing anything. Don’t touch any buttons at all. Look at the screen and notice everything out of the ordinary, and remember it or write it down. Then perhaps start cautiously pressing “OK” or “Cancel”, whichever seems safest. Try to develop a reflex reaction – if a computer does anything unexpected, freeze.

If you manage to get out of the problem, whether by closing down the affected program or by rebooting the computer, a good thing to do is to try to make it happen again. We like problems that can be reproduce more than once.

Providing your own diagnosis might be helpful sometimes, but always state the symptoms. The diagnosis is an optional extra, and not an alternative to giving the symptoms. Equally, sending a modification to the code to fix the problem is a useful addition to a bug report but not an adequate substitute for one.

If we ask you for extra information, don’t make it up! We can’t help if the problem or order of actions doesn’t exist.

Using your intelligence to help us is fine. Even if your deductions are wrong, we are grateful that you at least tried to make life easier. But report the symptoms as well, or you may well make life much more difficult instead.



“So then I tried . . .”

Say “intermittent fault” to any programmer and watch their face fall.

Most intermittent faults are not truly intermittent. Most of them have some logic somewhere. Some might occur when the machine is running out of memory, some might occur when another program tries to modify a critical file at the wrong moment, and some might occur only in the first half of every hour!

Also, if you can reproduce the bug but we can’t, it could very well be that our computer and your computer are different in some way and this difference is causing the problem.

We want to know anything you can find out about the problem. Try it on another machine, perhaps. Try it twice or three times and see how often it fails. If it goes wrong when you’re doing serious work but not when you’re trying to demonstrate it, it might be long running times or large files that make it fall over. Try to remember as much detail as you can about what you were doing to it when it did fall over, and if you see any patterns, mention them. Anything you can provide is always helpful. Even if it’s only probabilistic (such as “it tends to crash more often when Word is running”), it might not provide direct clues to the cause of the problem, but it might help us reproduce it.

Most importantly, we want to be sure of whether we are dealing with a true intermittent fault or a machine-specific fault. When this happens we will want to know lots of details about your computer, so we can work out how it differs from ours. One thing you should definitely be ready to provide is version numbers. The version number of the program itself, and the version number of the operating system and probably the version numbers of any other programs that are involved in the problem.


“So I loaded the disk on to my Windows . . .”

Writing clearly is essential in a bug report. We are not mind readers and can’t tell what you meant, when this happens you haven’t told us about a bug or problem.



Summary

 

This bug report guide has been adapted from work by Simon Tatham. The original work was released under the OpenContent Licence.