Bug tracking is a broad topic; various aspects of it are discussed throughout this book. Here I'll try to concentrate mainly on setup and technical considerations, but to get to those, we have to start with a policy question: exactly what kind of information should be kept in a bug tracker?
The term bug tracker is misleading. Bug tracking systems are also frequently used to track new feature requests, one-time tasks, unsolicited patches—really anything that has distinct beginning and end states, with optional transition states in between, and that accrues information over its lifetime. For this reason, bug trackers are also called issue trackers, defect trackers, artifact trackers, request trackers, trouble ticket systems, etc. See Appendiks B, Free Bug Trackers for a list of software.
In this book, I'll continue to use "bug tracker" for the software that does the tracking, because that's what most people call it, but will use issue to refer to a single item in the bug tracker's database. This allows us to distinguish between the behavior or misbehavior that the user encountered (that is, the bug itself), and the tracker's record of the bug's discovery, diagnosis, and eventual resolution. Keep in mind that although most issues are about actual bugs, issues can be used to track other kinds of tasks too.
The classic issue life cycle looks like this:
Someone files the issue. They provide a summary, an initial description (including a reproduction recipe, if applicable; see “Treat Every User as a Potential Volunteer” in Kapitel 8, Managing Volunteers for how to encourage good bug reports), and whatever other information the tracker asks for. The person who files the issue may be totally unknown to the project—bug reports and feature requests are as likely to come from the user community as from the developers.
Once filed, the issue is in what's called an open state. Because no action has been taken yet, some trackers also label it as unverified and/or unstarted. It is not assigned to anyone; or, in some systems, it is assigned to a fake user to represent the lack of real assignation. At this point, it is in a holding area: the issue has been recorded, but not yet integrated into the project's consciousness.
Others read the issue, add comments to it, and perhaps ask the original filer for clarification on some points.
The bug gets reproduced. This may be the most important moment in its life cycle. Although the bug is not actually fixed yet, the fact that someone besides the original filer was able to make it happen proves that it is genuine, and, no less importantly, confirms to the original filer that they've contributed to the project by reporting a real bug.
The bug gets diagnosed: its cause is identified, and if possible, the effort required to fix it is estimated. Make sure these things get recorded in the issue; if the person who diagnosed the bug suddenly has to step away from the project for a while (as can often happen with volunteer developers), someone else should be able to pick up where she left off.
In this stage, or sometimes the previous one, a developer may "take ownership" of the issue and assign it to herself (“Distinguish clearly between inquiry and assignment” in Kapitel 8, Managing Volunteers examines the assignment process in more detail). The issue's priority may also be set at this stage. For example, if it is so severe that it should delay the next release, that fact needs to be identified early, and the tracker should have some way of noting it.
The issue gets scheduled for resolution. Scheduling doesn't necessarily mean naming a date by which it will be fixed. Sometimes it just means deciding which future release (not necessarily the next one) the bug should be fixed by, or deciding that it need not block any particular release. Scheduling may also be dispensed with, if the bug is quick to fix.
The bug gets fixed (or the task completed, or the patch applied, or whatever). The change or set of changes that fixed it should be recorded in a comment in the issue, after which the issue is closed and/or marked as resolved.
There are some common variations on this life cycle. Sometimes an issue is closed very soon after being filed, because it turns out not to be a bug at all, but rather a misunderstanding on the part of the user. As a project acquires more users, more and more such invalid issues will come in, and developers will close them with increasingly short-tempered responses. Try to guard against the latter tendency. It does no one any good, as the individual user in each case is not responsible for all the previous invalid issues; the statistical trend is visible only from the developers' point of view, not the user's. (In “Pre-Filtering the Bug Tracker” later in this chapter, we'll look at techniques for reducing the number of invalid issues.) Also, if different users are experiencing the same misunderstanding over and over, it might mean that that aspect of the software needs to be redesigned. This sort of pattern is easiest to notice when there is an issue manager monitoring the bug database; see “Issue Manager” in Kapitel 8, Managing Volunteers.
Another common life cycle variation is for the issue to be closed as a duplicate soon after Step 1. A duplicate is when someone files an issue that's already known to the project. Duplicates are not confined to open issues: it's possible for a bug to come back after having been fixed (this is known as a regression), in which case the preferred course is usually to reopen the original issue and close any new reports as duplicates of the original one. The bug tracking system should keep track of this relationship bidirectionally, so that reproduction information in the duplicates is available to the original issue, and vice versa.
A third variation is for the developers to close the issue, thinking they have fixed it, only to have the original reporter reject the fix and reopen it. This is usually because the developers simply don't have access to the environment necessary to reproduce the bug, or because they didn't test the fix using the exact same reproduction recipe as the reporter.
Aside from these variations, there may be other small details of the life cycle that vary depending on the tracking software. But the basic shape is the same, and while the life cycle itself is not specific to open source software, it has implications for how open source projects use their bug trackers.
As Step 1 implies, the tracker is as much a public face of the project as the mailing lists or web pages. Anyone may file an issue, anyone may look at an issue, and anyone may browse the list of currently open issues. It follows that you never know how many people are waiting to see progress on a given issue. While the size and skill of the development community constrains the rate at which issues can be resolved, the project should at least try to acknowledge each issue the moment it appears. Even if the issue lingers for a while, a response encourages the reporter to stay involved, because she feels that a human has registered what she has done (remember that filing an issue usually involves more effort than, say, posting an email). Furthermore, once an issue is seen by a developer, it enters the project's consciousness, in the sense that that developer can be on the lookout for other instances of the issue, can talk about it with other developers, etc.
The need for timely reactions implies two things:
The tracker must be connected to a mailing list, such that every change to an issue, including its initial filing, causes a mail to go out describing what happened. This mailing list is usually different from the regular development list, since not all developers may want to receive automated bug mails, but (just as with commit mails) the Reply-to header should be set to the development mailing list.
The form for filing issues should capture the reporter's email address, so she can be contacted for more information. (However, it should not require the reporter's email address, as some people prefer to report issues anonymously. See “Anonymity and involvement” later in this chapter for more on the importance of anonymity.)
Make sure the bug tracker doesn't turn into a discussion forum. Although it is important to maintain a human presence in the bug tracker, it is not fundamentally suited to real-time discussion. Think of it rather as an archiver, a way to organize facts and references to other discussions, primarily those that take place on mailing lists.
There are two reasons to make this distinction. First, the bug tracker is more cumbersome to use than the mailing lists (or than real-time chat forums, for that matter). This is not because bug trackers have bad user interface design, it's just that their interfaces were designed for capturing and presenting discrete states, not free-flowing discussions. Second, not everyone who should be involved in discussing a given issue is necessarily watching the bug tracker. Part of good issue management (see “Share Management Tasks as Well as Technical Tasks” in Kapitel 8, Managing Volunteers) is to make sure each issue is brought to the right peoples' attention, rather than requiring every developer to monitor all issues. In “No Conversations in the Bug Tracker” in Kapitel 6, Communications, we'll look at ways to make sure people don't accidentally siphon discussions out of appropriate forums and into the bug tracker.
Some bug trackers can monitor mailing lists and automatically log all emails that are about a known issue. Typically they do this by recognizing the issue's identifying number in the subject line of the mail, as part of a special string; developers learn to include these strings in their mails to attract the tracker's notice. The bug tracker may either save the entire email, or (even better) just record a link to the mail in the regular mailing list archive. Either way, this is a very useful feature; if your tracker has it, make sure both to turn it on and to remind people to take advantage of it.
Most issue databases eventually suffer from the same problem: a crushing load of duplicate or invalid issues filed by well-meaning but inexperienced or ill-informed users. The first step in combatting this trend is usually to put a prominent notice on the front page of the bug tracker, explaining how to tell if a bug is really a bug, how to search to see if it's already been filed, and finally, how to effectively report it if one still thinks it's a new bug.
This will reduce the noise level for a while, but as the number of users increases, the problem will eventually come back. No individual user can be blamed for it. Each one is just trying to contribute to the project's well-being, and even if their first bug report isn't helpful, you still want to encourage them to stay involved and file better issues in the future. In the meantime, though, the project needs to keep the issue database as free of junk as possible.
The two things that will do the most to prevent this problem are: making sure there are people watching the bug tracker who have enough knowledge to close issues as invalid or duplicates the moment they come in, and requiring (or strongly encouraging) users to confirm their bugs with other people before filing them in the tracker.
The first technique seems to be used universally. Even projects with huge issue databases (say, the Debian bug tracker at http://bugs.debian.org/, which contained 315,929 issues as of this writing) still arrange things so that someone sees each issue that comes in. It may be a different person depending on the category of the issue. For example, the Debian project is a collection of software packages, so Debian automatically routes each issue to the appropriate package maintainers. Of course, users can sometimes misidentify an issue's category, with the result that the issue is sent to the wrong person initially, who may then have to reroute it. However, the important thing is that the burden is still shared—whether the user guesses right or wrong when filing, issue watching is still distributed more or less evenly among the developers, so each issue is able to receive a timely response.
The second technique is less widespread, probably because it's harder to automate. The essential idea is that every new issue gets "buddied" into the database. When a user thinks he's found a problem, he is asked to describe it on one of the mailing lists, or in an IRC channel, and get confirmation from someone that it is indeed a bug. Bringing in that second pair of eyes early can prevent a lot of spurious reports. Sometimes the second party is able to identify that the behavior is not a bug, or is fixed in recent releases. Or she may be familiar with the symptoms from a previous issue, and can prevent a duplicate filing by pointing the user to the older issue. Often it's enough just to ask the user "Did you search the bug tracker to see if it's already been reported?" Many people simply don't think of that, yet are happy to do the search once they know someone's expecting them to.
The buddy system can really keep the issue database clean, but it has some disadvantages too. Many people will file solo anyway, either through not seeing, or through disregarding, the instructions to find a buddy for new issues. Thus it is still necessary for volunteers to watch the issue database. Furthermore, because most new reporters don't understand how difficult the task of maintaining the issue database is, it's not fair to chide them too harshly for ignoring the guidelines. Thus the volunteers must be vigilant, and yet exercise restraint in how they bounce unbuddied issues back to their reporters. The goal is to train each reporter to use the buddying system in the future, so that there is an ever-growing pool of people who understand the issue-filtering system. On seeing an unbuddied issue, the ideal steps are:
Immediately respond to the issue, politely thanking the user for filing, but pointing them to the buddying guidelines (which should, of course, be prominently posted on the web site).
If the issue is clearly valid and not a duplicate, approve it anyway, and start it down the normal life cycle. After all, the reporter's now been informed about buddying, so there's no point wasting the work done so far by closing a valid issue.
Otherwise, if the issue is not clearly valid, close it, but ask the reporter to reopen it if they get confirmation from a buddy. When they do, they should put a reference to the confirmation thread (e.g., a URL into the mailing list archives).
Remember that although this system will improve the signal/noise ratio in the issue database over time, it will never completely stop the misfilings. The only way to prevent misfilings entirely is to close off the bug tracker to everyone but developers—a cure that is almost always worse than the disease. It's better to accept that cleaning out invalid issues will always be part of the project's routine maintenance, and to try to get as many people as possible to help.
See also “Issue Manager” in Kapitel 8, Managing Volunteers.