Task Tracker Requirements
- 1 Introduction
- 2 Basic principles
- 3 Ideal implementation
- 3.1 Data model
- 3.2 Supported activities
- 3.3 Differences with Launchpad
- 3.4 Open questions
Why yet another bug tracker ? Why not keep Launchpad ? Why not use X ?
Bug and feature tracking, or more generally task tracking, is closely linked to your workflow. Each tool out there enforces their own view on how to track bugs and/or features. You generally choose a tool that enforces a workflow that is acceptable to your project. But if the tool you use is not specifically tailored to your workflow, then it will do extra things and be confusing, or require you to do extra things to keep it consistent with your workflow. For example, suppose your bug triaging activity is about setting a priority to a bug. If you have a status that has "confirmed" and "triaged", which one should you set ? And if you can set priority independently of that triaging status, you end up with bugs that are triaged without priority, or bugs that are untriaged but with a priority set. At the other end of the spectrum, there are tools that don't enforce any workflow (think github issues) -- keeping those consistent is also a daunting task, and they generally don't handle a collection of related projects that well (each project is generally seen as independent).
We originally based our workflow on Launchpad, which is tailored to Ubuntu distribution workflows. It served us well, but OpenStack grew to the point where the small differences in workflows are getting more annoying, once multiplied by the number of persons involved in the project. At the same time, Launchpad development stalled, leaving us with little chances to improve the tool to suit our needs.
To help us in evaluating future options, this document will detail our requirements for a task tracking tool (features and bugs) and propose a reference implementation.
The main principle is that a single story (bug or feature) is broken down in multiple tasks. This is one thing that Launchpad Bugs really did right. Tasks are units of work that affect a specific project and a specific branch. They can be affected to different people. But tasks are always presented as part of a story, which is where the discussion happens. Tasks let you have bugs or features that affect multiple projects and branches, which is critical for OpenStack, and track them much more conveniently than just by using a single table and dependencies between single-dimensional items.
The second principle is that bugs and features are fundamentally different. Bugs can have tasks for stable branches, while features only have tasks for the version under development. Features and bugs show up in separate lists in release summary pages. Features are not a "wishlist" priority bug.
The third principle is that while bugs and features are different, they can be tracked with the same tools, because they have a lot in common. There is no need in maintaining them completely separately.
- Milestones (1:n, ordered by date)
- Type: Feature or Bug
- Importance ([Untriaged, NeedInfo?, Invalid, must have, should have, could have, wishlist] for features, [Untriaged, needInfo?, Invalid, Critical, High, Medium, Low] for bugs)
- Tasks (1:n, ordered?) -- can have multiple tasks for the same project/series combination
- Title (by default "series backport" for a bug, a number for a feature)
- Project (or "other")
- Status (Todo, under development (?), in review, landed in branch, blocked(?))
- Target milestone (+ landed milestone ?)
- Proposed change link(s ?)
- Release tracking flag (set by drivers)
- Whiteboard (?)
- Visibility: Security bug or Public
- CVE references
- Comments (1:n, ordered by date)
- Attachments (or just enable links in description and comments ?)
Triaging is about looking into bugs and giving them an importance. This activity can be privileged or open to everyone. Bugs start out as Untriaged. If they require more information before being triaged, importance can be set to NeedInfo. If they are invalid, they should be set to Invalid. If not, pick an importance (Critical/High/Medium/Low). Maintaining all that activity under the same field (importance) avoids incoherence between a status (New/Incomplete/Invalid/Confirmed/Triaged) and the absence of importance set (Undetermined/Low/Med/High). If an importance is set, it is triaged.
Some bugs should be able to be marked as duplicates, although for performance reasons, it might make sense to keep it simple and just set them to Invalid with a link to the other bug in the comments.
Preparing a series roadmap
Series roadmapping is about triaging proposed features and making them appear in your PTL roadmap for the series under development. this activity should be restricted to PTLs and trusted lieutenants (the drivers group). Launchpad here uses two fields (series goal and milestone target) which is VERY confusing. The proposed model lets you triage blueprints the same way you triage bugs. But once triaged, you can also set the release radar flag, which will make the feature (or the bug) appear in the release tracking pages (see below). It's basically a way of saying "we care about this bug or this feature for the release under development", but much more flexible than series goal and milestone targets.
Tracking milestones objectives
Milestone and release pages are another area that Launchpad did well and we should emulate. Milestone objective pages show the list of features and bugs that are targeted to the current milestone under development (i.e. have the "release radar" flag set). The same summary pages can be reused for:
- Project milestone pages: Show all tasks (features, then bugs, on two separate lists) on release radar for a given milestone for a given project
- Release status: Show all features (by story or by task ?) on release radar for all milestones in a series for all integrated projects
Producing release pages
Release pages are another variation on the above summary page. They show all completed tasks for a given milestone (features, then bugs, on two separate lists), and should have space for uploading a deliverable (?). The final series release page shows all completed tasks for all milestones in a series.
Working on code
The developer can optionally set a task 'Under development', but otherwise, the task status will advance to in review or landed in branch automatically when a linked review is uploaded and merged in Gerrit.
Differences with Launchpad
The main difference is that we are using the same system to track features and bugs. That opens a whole world of goodness to features: they can span multiple projects, they can have comments, tasks can be used as work items, they can be invalid, have CVEs, duplicates...
The second is that we get rid of plenty of extra fields and values that are irrelevant to our workflow. Confirmed/ Triaged statuses being separate from priorities, the Fix released status that doesn't really help us... On the features side, approver, drafter, definition status, series goal (distinct from milestone target), linked branches and sprints... all gone.
We extend the BugTasks system to make it support work items (you can have several items targeted to the same product/series combination).
Using the release radar flag for editorial control instead of relying on a complex combination of propose for series, approve design, setting milestone target should prove much more efficient and intuitive.
Launchpad blueprints let complex dependencies be specified. Within a given story, my simple proposal is to have an array of tasks that can be arranged in any order. I don't think extra complexity would be warranted... But we could also have complex dependencies between stories (ala Launchpad blueprints).
Tasks and proposed changes
How do we track task completion vs. merged changes ? Currently we suppose that when a bug is mentioned in a change, we should set it to FixCommitted when it lands. But sometimes it's only a partial fix, and for blueprints that corner case would happen much much more often. It would be neat to have the tooling solve that as well.
One way to do it is to reference a specific task rather than a story in commit messages, but that could be unconvenient to specify. We could also require that each proposed change is a separate task, potentially even auto-creating tasks in a story when the commit just mentions the story. That would let us track precisely where we are in the implementation of a feature.
Features spanning multiple series
The framework actually lets you have features with tasks that span multiple series. I think that's OK... but that means release pages would show feature tasks rather than just features, which could be confusing. Or could mention incomplete features. Maybe that's just a UI problem (show partial blueprints differently).
The proposed model is tailored to the way everyone else behaves. In particular, I thought series and milestones would be defined for all projects rather than separately for each project. Series release pages also don't really make sense in a Swift world.