Measuring Partial Points

I measure Scrum teams using a spreadsheet inspired by Scott Downey's RoboScrum. One key indicator I watch for is partial work. To illustrate how this indicator is used, I will tell a story.

Sound Familiar?

A UI team of 6 members begins a sprint hoping to finish 35 story points worth of stories. They swarm effectively and near the end of their 2 week sprint, the team is so close to completing all of their stories. However, the team just knows two "in progress" stories are not going to get done.

The team shows up at the Sprint Review meeting bummed that they didn't get all their planned stories done. "I don't understand why we don't get credit for those stories, we were almost done!" the team complains to their ScrumMaster.

"Remember," the ScrumMaster reminds his team, "you do get credit!"

The ScrumMaster chuckles to himself as he reads the contorted faces of his team. "Velocity is the number we use to show how much we can get completely done. This is an important measurement for our product owner."

"But what about that 8 point story that only has a small amount of testing left? You mean we aren't going to get credit in our velocity for what we completed in that story? We probably only have 1 point left!"

"I understand," the ScrumMaster accepts, "but did we get the story done? No."

The ScrumMaster explains that credit for this partially done story is an important indicator that that team can use in future planning. Together with their velocity, the team can use partial points to determine their total capacity during the sprint as a goal for future sprints.

Therefore, capacity is equal to a team's velocity plus the sum of partial points.

Capacity = Velocity + Partial Points

Why Measure Partial Points?

Not only do partial points allow us to figure out a team's capacity, in story points, but they also help us measure a team's focus factor. Scott Downey described focus factor in this way:

I, as a member of Leadership and not a member of a Scrum Delivery Team 

Need a way to measure how much of each Team's bandwidth results in deployable product, in a cross-team comparable way, 

So that I can actively help sub-optimized teams intelligently allocate resources.

The formula for focus factor is:

Focus Factor = Velocity ÷ Capacity

In summary, focus factor tells how close a team was to finishing 100% of their sprint. Teams that get around 80% in this category make me happy. Why 80%? Consider this statement by W. Edwards Deming:

Attempts to force non-deterministic systems to operate at greater than 80% efficiency will cause short bursts of stabilization followed by extreme periods of destructive and unpredictable variations from that goal.

Besides, a team with a repeated focus factor of 100% tells me they probably need to take a sprint and stretch to new heights. A team consistently less than 80% may be taking on too much, or putting too many stories into progress at once.

How do I Record Partial Points?

Here is a sample from my metrics spreadsheet:

The "Actuals" section contains the results from each sprint (one sprint per row). The "Partial Done" column in the "Actuals" section contains the sum of all partial points from that sprint. This value of partial points automatically feeds into my Summary view to show the team's capacity (velocity + partial) and the team's focus factor.

How do I Determine Partial Points?

Recording them is the easy part! Getting a team to quantify partial points requires more work. During a sprint review, I jump to stories that are not accepted as done and ask the team these questions:

1. What is left to be done with this story?
2. What type of work is required to get the remainder done? Database work? Nagging? Test case verification?
3. What work was completed?
4. What type of work was the completed work? Coding? Automated tests? Design? Nagging?
5. Compared to our reference stories (you have them right?), how would the team estimate the work they completed?
6. Compared to our reference stories, how would the team estimate the remaining work?

Question #5 tells you the number of partial points to capture for each story. The sum of these partial points is what is recorded above in the "Actuals" section.

Follow-on stories to spikes

One of my ScrumMasters presented the following scenario about spikes:

Our team lead asked me about this and I wasn’t sure how to answer him. So I hope you can help. 

We have a case where we have a spike story in our sprint. We have assigned some points to the story, say 3. After work on the spike is done we will create a new story based on what we learn from the spike. We don’t know the number of points to assign to the new story until the spike is done. 

This isn’t a problem if we put the new story in the following sprint. The problem is what to do if the new story is in the same sprint as the spike. 

Here’s my guess as to how we handle this situation and get all the ‘done’ credit we deserve: We assign the max amount of points we think the new story will take, say 8. Then if the story turns out to be less, say 3, then we pull in 5 points worth of stories from the back log. That will keep our total sprint points matched with our prediction. 

Is there a better way to handle this situation?

So, a team has two stories A (the spike) and B. B should NOT be pulled into a sprint until it is "ready." Remember, "ready" means that the story has acceptance criteria and an estimate in story points. Until A (the spike) is done, you don't know the acceptance criteria and estimate for B, and is therefore, not ready.

For a great exposition on the impact of ready stories see this article by Carsten Jakobsen and Jeff Sutherland: Scrum and CMMI - Going from Good to Great (Are you ready-ready to be done-done?).

If you want A and B done in the same sprint, the only logical strategy is to plan for A during your sprint planning and leave B on the product backlog. Once the spike is done, the team needs to work with the product owner to determine the acceptance criteria for B and put an estimate on it - making it "ready." Then, B will be "ready" and can be "adopted" into the sprint if the team feels like they can do B plus the rest of their planned stories.

Assigning the max amount of points to B is just sand-bagging an unknown risk. Avoid that. Teams are more productive when they plan stories that are ready because they don't have to go back and forth during the sprint to get more clarity on stories with the product owner. 

Teams that adopt stories consistently show that they should plan to do a higher target velocity. Teams that have consistent unplanned work show that they are not prepping the backlog effectively. There is a sweet spot you have to find as a team. 

This A (spike)/B pattern is a rare occurrence for teams. Training the product owner to know that spikes run ahead of the real story implementation helps remedy this dysfunction.