To ask a software developer for an estimate is often a source of serious friction—some people would rather have their teeth drilled.

Estimating technology delivery is one of the more challenging problems we face in tech. Have you ever felt it's easier to just double the estimate, and maybe double it again, because no matter what else you try you cannot get accurate numbers?

Don't just assume accurate estimates are impossible.

I believe the heart of this problem centers around how much the developer feels they are empowered in the actual estimating process. Have you ever heard or seen the following dialog? Consider how it could effect the estimating cycle:

        Chuck, the project coordinator asks John, the developer, “How long will it take to make this adjustment? It should be pretty easy, right?”
        John feels something is missing in what is requested, but hasn't been given time to collect his thoughts, and is already swamped with several things, responds with a hesitant, “I’m not quite sure, maybe a few weeks? A month?”
        Chuck grimaces, “Well, we told the client it'd be next Friday, can we make that happen?”
        John's lips are pinched as he answers, “Umm, that’s a lot of work.”
        “But this client is really important to us, millions on the line, so whatever we can do,” Chuck adds while leaning forward on the desk.
        Something snaps inside John as he stops caring, “Fine, sure, ok.”
        Chuck smiles, feeling he has accomplished his goal of moving things forward and thinks he should end it by being a great leader and being positive, “Great, you are awesome!”
        John stares back at the code on his computer monitor, thinking: whatever, I'll just get done what I can, and I'm out of here at five sharp.

Or a simpler version:

        Chuck announces, “Next week’s tasks are on the queue. John, we have this issue, it’s scoped for 40 hours, can you get it done next week.”
        John, having given up, thinks: well, I didn’t make the estimate of 40, it’ll probably not be done, but I can hack up something, even if it’s incomplete, it'll be QA's problem then, and answers, “Sure.”
        Chuck, “Great, moving on…”

This dialog is toxic, yet it's more common than we may think. The reason it's a problem is that it geld's the developer's ownership, and it's Chuck's fault. If you have experienced this as a planner, but you think your devs just cannot ever give good estimates, think back to how that pattern might have been created before now.

But does this mean we just stop estimating work? Or stop pressing on the developers to better scope what they have in the queue?

No.

Measurement

“Measurement is the first step that leads to control and eventually to improvement. If you can’t measure something, you can’t understand it. If you can’t understand it, you can’t control it. If you can’t control it, you can’t improve it.”

― H. James Harrington

The best engineers can scope their work and usually hit it. There are many tools available, and some of it is simply practice to improvement. In sports, the junior teams usually don't track all of the details of a player's performance, but by the time a player reaches professional levels, they are expected to be measured and continue to improve on their ability to perform.

The same applies to developers, and good developers can estimate and achieve their estimates, without having to double, and double again an estimate. If you find your estimates are off, then consider what was missed in the estimate? Perhaps testing? Delivery? Or something else often wrapped up in the umbrella of "technical debt."

Ownership

For a developer, getting better at estimating and scoping also means they need to feel ownership. In part, this is also about Delegating with Success. But this is also about trust.

For ownership to work, trust has to exist. And for trust to work, we have to fix this dialog. But the dialog cannot be fixed if tools and processes are an impediment (so-called "waterfall" type processes). This is why the first statement of Agile is simply:

Individuals and interactions over processes and tools

This means sometimes it makes sense to set aside the tools, the books, and the processes, and just build relationships, trust, and have dialog that is beneficial. Use notes taped to a wall for a few iterations, just focus on getting the work to flow, without tools getting in the way.

Simplify

I find, when trying a process reset, it is very beneficial to focus on three questions on a weekly basis:

What did you do last week (completed, planned and shoulder-taps)
What was missed -- and why?
What are you going to /finish/ this week? (they choose)

It is a variant of the popular Progress/Problems/Plan reporting (PPP), but some nuances exist to specifically address the software development problems.

Set aside all tools for a moment, pretend you don't know anything about any tools, and just consider how a weekly dev planning meeting could be run while focusing on these questions:

Before meeting with devs: Curate a few lists of work (whatever works for you: backlog, todo, etc), and curate them before the meeting so you are not shuffling things in the middle of a meeting.
Have two views ready to talk through with the team:
a. What did everybody accomplish last week (including shoulder taps), and how many hours were spent on each
b. What can we work on this week (including in-progress rollover things)
Step through the first view with the team and discuss the accomplishments (at a high level). This might go something like:

        “Josh, looks like you put in 35 hours on these six tasks, that's awesome, any problems? No, cool, great work!”
        “Lany, wow, 60 hours? That's a lot, what happened? Anything we can do to avoid this in the future? Cool, great work!”
        “Sue, 15 hours are reported, did we miss something? Ok, cool, yeah, get those shoulder taps recorded too. Appreciate your work!”
        “And Voy, you only had 5 hours but I know you were on vacation this week, so wow, sorry to drag you in!”

Then switch to the what’s coming up view:

        “Josh, we have a carryover from last week--let's try to break that down better next time so we don't have any carryover work. How much is left on this? Cool. What else can you pull from the backlog so you are hitting 30 planned hours?” (Josh picks a story or two until he's in the zone)
        “Lany, you said you were interested in this other Story, do you want to do it this week? You think it'll be longer than 30-40? Then let's break it into smaller pieces. What are you absolutely confident you can finish this week? Great!”
        “Sue, you are on production this week so let's put you only at 10 hours planned.”
        “Voy is out again, no work!”

At one place I worked we floundered with problems, processes, and tools. We had the classic problem (but I believe toxic): double every estimate, then double it again! No matter what tool we used it didn’t help. I ended up setting aside all tools and just using a simple list of work we made in a google doc — much like simply using notes on the wall.

It was painful, but it simplified a lot, and we did this for a few iterations, focusing on the above questions until we had everybody in that mindset. Then we went back to the tools.

If you ever have to double, and then double again an estimate, it might be a red flag that you are missing something in your planning process.

There are two key parts of this dialog that are not accounted for above, which also help the estimating:

Prior to the meeting you are also meeting with devs and they are breaking the work down themselves, and scoping it (don’t do this during the planning meeting)
A Deliverable Breakdown Structure (DBS) — when you see the Dev being resistant to scope a change, don't just say: well, they are stuck in the weeds, or don't understand the business. Instead, empathize. There is probably something in their gut that they cannot put words to, so this is when I pull out the DBS, to help them elevate visibility to it.

Also, consider the Agile Deceit and Activator, a quick guide to using Design Thinking in Technology to break down these very problems.