Monday, September 19, 2011

Measuring Partial Points

I measure Scrum teams using a spreadsheet inspired by Scott Downey's RoboScrum. One key indicator I watch for is partial work. To illustrate how this indicator is used, I will tell a story.

Sound Familiar?

A UI team of 6 members begins a sprint hoping to finish 35 story points worth of stories. They swarm effectively and near the end of their 2 week sprint, the team is so close to completing all of their stories. However, the team just knows two "in progress" stories are not going to get done.

The team shows up at the Sprint Review meeting bummed that they didn't get all their planned stories done. "I don't understand why we don't get credit for those stories, we were almost done!" the team complains to their ScrumMaster.

"Remember," the ScrumMaster reminds his team, "you do get credit!"

The ScrumMaster chuckles to himself as he reads the contorted faces of his team. "Velocity is the number we use to show how much we can get completely done. This is an important measurement for our product owner."

"But what about that 8 point story that only has a small amount of testing left? You mean we aren't going to get credit in our velocity for what we completed in that story? We probably only have 1 point left!"

"I understand," the ScrumMaster accepts, "but did we get the story done? No."

The ScrumMaster explains that credit for this partially done story is an important indicator that that team can use in future planning. Together with their velocity, the team can use partial points to determine their total capacity during the sprint as a goal for future sprints.

Therefore, capacity is equal to a team's velocity plus the sum of partial points.

Capacity = Velocity + Partial Points

Why Measure Partial Points?

Not only do partial points allow us to figure out a team's capacity, in story points, but they also help us measure a team's focus factor. Scott Downey described focus factor in this way:

I, as a member of Leadership and not a member of a Scrum Delivery Team 
Need a way to measure how much of each Team's bandwidth results in deployable product, in a cross-team comparable way
So that I can actively help sub-optimized teams intelligently allocate resources.

The formula for focus factor is:

Focus Factor = Velocity ÷ Capacity

In summary, focus factor tells how close a team was to finishing 100% of their sprint. Teams that get around 80% in this category make me happy. Why 80%? Consider this statement by W. Edwards Deming:
Attempts to force non-deterministic systems to operate at greater than 80% efficiency will cause short bursts of stabilization followed by extreme periods of destructive and unpredictable variations from that goal.

Besides, a team with a repeated focus factor of 100% tells me they probably need to take a sprint and stretch to new heights. A team consistently less than 80% may be taking on too much, or putting too many stories into progress at once.

How do I Record Partial Points?

Here is a sample from my metrics spreadsheet:


The "Actuals" section contains the results from each sprint (one sprint per row). The "Partial Done" column in the "Actuals" section contains the sum of all partial points from that sprint. This value of partial points automatically feeds into my Summary view to show the team's capacity (velocity + partial) and the team's focus factor.


How do I Determine Partial Points?

Recording them is the easy part! Getting a team to quantify partial points requires more work. During a sprint review, I jump to stories that are not accepted as done and ask the team these questions:
  1. What is left to be done with this story?
  2. What type of work is required to get the remainder done? Database work? Nagging? Test case verification?
  3. What work was completed?
  4. What type of work was the completed work? Coding? Automated tests? Design? Nagging?
  5. Compared to our reference stories (you have them right?), how would the team estimate the work they completed?
  6. Compared to our reference stories, how would the team estimate the remaining work?
Question #5 tells you the number of partial points to capture for each story. The sum of these partial points is what is recorded above in the "Actuals" section.


4 comments:

  1. Thanks so much for this post. Have you modified RoboScrum to work with 2-week sprints? If so would you share?

    ReplyDelete
  2. I think you left something out, or if not then I'm misunderstanding. If you have an 8 point story that doesn't make it because the team only completed 5 points worth, then 5 points goes into capacity for this sprint and 0 for velocity your way. But what about next sprint? If all 8 go into capacity there then you're overstating capacity.

    The mathematically correct thing to do for the next sprint's capacity would be to count only those points actually left to be worked on (presume the remaining 3 only), and still count all 8 for velocity. But then you could have capacity less than velocity and that can't be right.

    I have also heard of teams where no points are counted as velocity for the first sprint and only 3 as velocity in the second, but that understates velocity.

    Where I work we have decided that the benefit of this last bit of "accuracy" provided by partial points for capacity measurement is not worth the cost. So the way we count is that if you touch a story in a sprint and have to push it to the next sprint, all of its points go into capacity for the first sprint. And again in next sprint whether you finish it there or push it again. This overstates capacity (and therefore understates focus factor) a little. But let's face it, those are fuzzy numbers anyway and their trend is what is important.

    If your team is spending time thinking about how much capacity credit they deserve for a story they haven't completed, then I think they and the scrum master are missing the point. The aim is to complete stories.

    ReplyDelete
  3. I'm also looking for a copy of Roboscrum converted for a 2 week sprint, if anyone has successfully do this can they share it please?

    ReplyDelete
  4. Hi, I would also like a copy of Roboscrum for a 2 week sprint..

    ReplyDelete