Older blog entries for cm (starting at number 13)

Apologies to all for the long delay between these. Work has been hectic and off hours I need my rest.

Schedule Reviews Before Big Changes is the next and penultimate of the Defect Tracking Patterns.

Sometimes a bug or group of related bugs will result in a change that affects large sections of the codebase, or significant changes to one component. Best practices would encourage small, incremental changes and ContinuousIntegration, but sometimes, for reasons outside the team's control, a large set of changes must happen all at once.

Therefore:

Before the programmer commits the changes to the team source repository, assign the bug report(s) to a reviewer, typically a team lead, senior programmer, or module owner, for an extra-careful check. The reviewer can examine the proposed fix for compatibility, suitability, and overall conformance with direction of development. Reviewers should be timely in their responses, and reassign the reviewed bug back to the programmer with approval or suggestions for changes. Once reviewed, the programmer can go ahead and integrate the fix.

It's assumed the team is doing normal revies all the time, either formal, informal, or through pair programming. If the only reviews the team does are before big changes, that's an anti-pattern, and also a failure to follow the CoordinateEfforts rule.

In the continuing series of Defect Tracking Patterns, up next is Incidents and Faults.

A single defect in one piece of code can cause different apparent problems to different users. It is also possible that a single reported problem may be traceable of multiple different pieces of code that are similarly faulty.

Therefore:

Several reported failures that arise out of the same fault should be tracked together. Consider splitting a single bug with multiple causes into a bug for each fault, and relate them as above.

However:

The bug tracking software must support the ability to associate bugs as duplicates of and dependent on other bugs. Splitting a single bug or otherwise tracking related bugs without good support from the tool requires too much manual overhead.

Happy Holidays everyone. I hope the new year brings everyone more work and less stress.

Now that I am back from my vacation I will resume my posts on the Defect Tracking Patterns language. We've come a bit over halfway, and I've gotten some good feedback. Thanks to everyone who has commented!

Fixed is not Closed

A simple but often overlooked distinction in tracking bugs in software. After the code to fix a bug is checked in, it still isn't really done until it is verified. In the context of the development team receiving feedback from the customer, programmers who work on bugs need feedback to know if the changes fixed the bug as reported. Those changes need to be verified by a third-party and the work on the bug brought to resolution.

Therefore A fixed bug is not a closed bug. Once the programmer has made code changes and checked in the results, that programmer can mark the bug fixed, but cannot close it. The customer who entered the bug is responsible for verifying and closing the bug if it really is fixed. The resolution of a bug should be different from the status. "Will not fix" or "Unable to reproduce" are among the other resolutions besides fixed that can close a bug. The dispatcher should be especially alert to resolutions other than "fixed", there may be additional information that can change this.

The customer who submitted the bug should close it, or, if there is an dispatcher role, that can be where fixed and closing are coordinated.

The previous pattern, A Bug's Life Cycle, is closely related to this pattern.

I'm on vacation until after the first of the next year, woohoo! I'm leaving today to visit my family in Houston, so I'll probably not post again until after Christmas. This one will have to hold you until then.

A Bug's Life Cycle

The sixth pattern in the Defect Tracking Patterns language.

Identifying and fixing bugs involves several steps to coordinate as the bug changes states and assignees. Any team member with a stake in a bug needs to know what stage the bug is in at any given time. The customer will look to see when it has been fixed, the dispatcher will need to be able to determine when the bug needs re-assigning, and programmers will need to know when a fix is tested and accepted.

Therefore

Define and follow a life-cycle for bugs.

A bug can only be assigned to one person at a time. That bug may flow back to the reporter with a misdiagnosis or fix that doesn't work. The assignee may assign the bug to another programmer more familiar with the fault or for review. It may go back and forth from reporter to programmer a few times before it is fixed, retested, and resolved.

See also DocumentWork; CoordinateEfforts

An example Bug State Chart

A quick note to bjf, I also recently bought a Nikon FM3A and I love it. I chose it because of it's classic design and reliability, and I'm glad to see not everything is going digital.

The next pattern in Defect Tracking Patterns is the one that has generated the most commentary so far, often because of confusion over the terms.

Differentiate Priority and Severity

The effect of a bug on the software does not automatically correlate with the priority for fixing it. A severe bug that crashes the software only once in a blue moon for 1% of the users is lower priority than a mishandled error condition resulting in the need to re-enter a portion of the input for every user every time.

Therefore:

Define a range for Severity as "how bad does it hurt?" according to project requirements. One example of a range in use is

  • Enhancement: New features
  • Low: Improvement to existing code, e.g. performance enhancement, or problems with an easy workaround
  • Normal: Broken or missing functionality
  • High: Problems causing crashes, loss of data, severe performance problems or excessive resource use.
  • Blocker: Problems that prevent testing or development work

Define Priority according to business value and project timelines. High priority bugs should be first to be fixed.

Importance and Destructiveness are alternate words which may be clearer to some.

An example of the difference and interaction is in bugs that are purely cosmetic problems, misspellings in dialogs, redraw issues, etc. These can be priority 1 fairly often, because they are frequently very annoying to users, and fixing them is generally easy and doesn't destabilize things.

Track priority and severity separately, then triage appropriately. It helps to have input from others on the team on priority. The importance of a bug is a project decision, different from the bug's perception by the Customer. In some cases it makes sense to track Urgency, the customer's point of view, separately.

The Developer, when determining what to work on next, would pick the bugs with the highest priority, regardless of severity. The Dispatcher is responsible for setting these appropriately, especially in the case where the Customer marks everything Very Severe and Highest Priority.

Extreme Programmers may see a resemblance to Story Cards and the Planning Game.

Coordinate Efforts through Defect Lifecycles

The next pattern in the Defect Tracking Pattern Language deals with the context of a development team that may be geographically dispersed or otherwise not able to sit together in an XP way, but has a sense of collective or at least weak code ownership. The product being worked on is large an complex enough that a defect tracking tool is need and used.

Untracked bugs can lead to wasted or duplicate effort. The wrong programmer attempts to fix it or multiple programmers all work on different solutions at the same time without coordination. Or a programmer works on an already-fixed bug not knowing that the fix is in. The bug isn't really a bug, it's a new piece of functionality, leading to scope creep. Low priority, easy, or "fun" bugs get fixed before the higher priority bugs.

Therefore:

Define and use a bug lifecycle. Assign bugs to one person at a time, and give that individual responsibility for resolving it or assigning it elsewhere.

As the customer, watch your bugs and respond to queries for additional information or candidate fixes. If a bug you reported is no longer an issue, close it. If you are using a bug tracking system that allows it, you may watch bugs you do not own but have a stake in. As the developer, update your bugs when you work on them. If you get stuck, assign them to another developer. If you need additional information or want comments from the submitter, assign it back to them with appropriate status.

Identify someone to play the Dispatcher role, and modify priority and assignments as necessary. The Dispatcher role can streamline the process of moving a bug through its lifecycle.

However

Adhering tightly to this pattern requires discipline in using the bug tracking system. The Use a Tracking Tool and Document Work patterns are intimately tied to this pattern's effectiveness.

See: DocumentWork

Next on the list of Defect Tracking Patterns is the bug tracking tool and usage as a kind of history-keeping system. At the moment it's one of the weaker patterns -- comments welcome.

Document Work

A bug report has a life and grows with changes. Bug state is not static, and tricky defects can take several iterations of work to find and fix properly. In the context of this ongoing work, knowing what changed and why, and what worked or didn't, help maintain a sense of history to the code.

Therefore

Document all work on a problem in the bug tracking tool. Use the description and comments field to make note of important issues and facts. Attach files with error output, screenshots, or even snippets of code patches to document what was done and why. Records of what was worked on and resolved represent a knowledge base.

The related idea of micro-decision awareness asks only that when making a decision, however small, you are aware of the fact. Recording that decision in the bug tracking software preserves that decision.

14 Dec 2002 (updated 14 Dec 2002 at 17:05 UTC) »

Sorry for the silence, Wednesday I went home early and I've been home since. I picked up a wicked cold and haven't been able to think straight.

Thanks robocoder for taking a look at the whole of DefectTrackingPatterns, I welcome your comments. Until I get my head clear, here's a short list of my brain dump of things that a good bug tracking system ought to have.

  • Users can split items (e.g. when one item actually describes two separate issues) or combine them (e.g. when several contributors submit identical items)
  • Users can associate items in groups and dependencies.
  • Old discussions or issues archived, indexed, and searched. When and how does the system and/or its users identify submissions or discussions that are re-raising old topics or issues?
  • Issues, and changes in the state of issues, are connected to the use of other tools, such as checking source code in or out
  • Users are able to perform intelligent searches on issues, i.e. something more sophisticated than searching for keywords in the bodies of issues
  • URL-style link to defect in other documents, for example http://bugzilla.mozilla.org/show_bug.cgi?id=100000
  • Allow defects to have dependencies on other defects, use to create "tracking defects" and "release requirements" defects that refer to other defects that must be closed before the parent can be closed.
  • Allow users who are neither the submitter or assigned developer or admin to put themselves on a list of users to CC email on changes (watch list)
  • Clearly differentiate between defects and RFEs
  • Able to identify and push issues that are really support questions to the appropriate persons/tools
  • Use with any web browser on any O/S
  • Allow anyone on the email-on-change list to adjust which events they wish to get email about; then the system can by default be configured to email on all non-trivial events and users can turn the noise level down

Another installment on Defect Tracking Patterns, this one discusses how an automated bug tracking tool can help programmers and other development team members keep tabs on what their assigned tasks are.

Defect Tracking As Automated TODO List

Programmers and testers need a way to know what tasks they are responsible for and what priority those tasks have.

Therefore:

The defect tracking system should have sort and filter features that allow maintaining an automated TODO list. Each item in the bug tracking system is assigned to one person and has a priority associated with it. The list of bugs becomes a personal, automatically updated and prioritized, TODO list.

As a developer, filter on Assigned To to see your responsibilities. Sort by Priority to track the top problems that need to be worked on first.

The Dispatcher's role in assigning and prioritizing bug reports helps to keep things flowing during the development process.

Continuing the series of diary entries on Defect Tracking Patterns

This pattern is named Use a Tracking Tool.

Post-it notes, programmer's memories, even email, are all unreliable repositories for bug reports and feature requests. All of them are also difficult to coordinate on a team of more than 2 people working together for more than a few days.

Therefore:

Use an automated tracking tool. The tool doesn't have to be sophisticated -- it can be a spreadsheet in simple cases -- but it must be shared.

Bug tracking software has features to help manage work. Users can create filters and sorts to assist workflow. Filter on the field Detected By with your id to show the bugs you've filed. Filter on Assigned To to see your responsibilities. Sort by Severity to track the top problems.

4 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!