Archive for the ‘Estimating’ Category.

NoEstimates at Oredev 2013

By Woody Zuill

I was invited to present my ideas about “No Estimates” at Oredev 2013 in Malmo, Sweden.  It was an exciting and wonderful experience for me (to say the least), and perhaps someday soon I’ll post more about it.

Until then, here is a link to a video of the session I presented there about #NoEstimates:

NoEstimates at Oredev 2013

Please let me know what you think, if you would.

The NoEstimates Hashtag

In Twitter, we use the convention of “hashtags” to connect a tweet with a theme or topic so that it is easy (or at least a bit easier) to search and find tweets for topics we want to follow.

Several people have blamed me for coining the “#NoEstimates” hashtag.  It might be true – I’m not sure.  It is possible that I was the first to use this hashtag for Software Development – a search of Twitter history on it shows that, as far as I can tell.

[UPDATE: I found the following post from Aslak Hellesøy – @aslak_hellesoy dated Feb 10, 2010 that used this hashtag

@obie at #speakerconf: “Velocity is important for budgeting”. Disagree. Measuring cycle time is a richer metric. #kanban #noestimates.]

SO… What is the Topic of the #NoEstimates Hashtag?

Here it is, as defined by me.  Others probably have their own idea:

#NoEstimates is a hashtag for the topic of exploring alternatives to estimates [of time, effort, cost] for making decisions in software development.  That is, ways to make decisions with “No Estimates”. 

I’ll probably modify this a bit over time – I put this together quickly as a starting point.

Also, A Word About Estimates

Estimates are typically and pervasively used in the decision-making process throughout software development organizations and efforts – both in “phased/waterfall” and “Agile” methodologies.  They are ubiquitous – it is rare to find a project/organization/effort where it isn’t simply accepted that “we must have estimates – it is the only way”.

Additionally, for a long time I’ve been noticing that almost every training, book, conference session, article, or blog on “Agile” software development has a very heavy focus on estimating.  I’m fine with important things taking a lot of our attention – they are important, after all. Are estimates that important?

I’ve also noticed over the years that many organizations have a number of dysfunctions in how estimates are handled, and the overall decision making process that depends on estimates.  Have you noticed any dysfunctions?

So… I started doing “5 why sessions” or similar exercises at conferences and user groups, and other investigations into the reasons for this, and the dysfunctions that people have been experiencing.  I also started discussing alternative ways to manage software development efforts without using estimates. I’ve been working without estimates for years for many decisions that are typically “estimate-driven”  in software development management, and some people are interested in how to do this.

It’s important to ask ourselves questions such as: Do we really need estimates? Are they really that important?  Are there options? Are there other ways to do things? Are there BETTER ways to do thing?

If we are willing to simply play the “devil’s advocate”, and imagine (or at least temporarily pretend) that perhaps there are better ways to make decisions, it can lead to more open thinking and clarity as to why we “believe the things we believe”.  Why not question the things we hold so dearly?

There ya go!  Have Fun!

 

 

My Customers Need Estimates, What Do I do?

It seems a lot of people have an argument against the “No Estimates” idea that goes something like this:

“Our Customers Need Estimates*”

There are many variations on this, but they typically come down to the same thing: We need customers, customers need estimates, therefore, we do estimates. The following are a few generic examples of this idea:

  • No customer will agree to hire us to write custom software for them unless we give them an “estimate”.
  • The reality of business is that we must give estimates to our customers so we can get work.
  • We must be good at estimates because our competitors are giving estimates.
  • The customer want’s to know if we can do their work at a price they can afford
  • The customer needs to know if they can make a profit based on the amount their software will cost, so they need an estimate.

How Can I Get These Customers Without Estimates?

This is a very easy question to answer: You probably can’t get these customers without estimates.  They want estimates – it’s part of the way they think. You probably MUST DO ESTIMATES for these customers.  That is, you don’t tell these customers “We don’t do estimates” if you want to work with them.

If you have chosen to work with the sort of customer that requires an estimate (or price) then give them what they want: an estimate (or price).  Your main job is to figure out a way to be able to make money taking on work for this sort of customer.  I understand that. You understand that. What more do you need to know?

A few points you might want to consider:

  • Are you willing to give an estimate/price for free?
  • Who pays for the estimates for jobs you don’t get?
  • How much extra do you need to “pad” the work for the “unknows”? Why should your customer pay for that?
  • Are you going to give a range?: “It will cost you no less than this but no more than that”.  Will you guarantee that?
  • If it costs you more to make the software than you estimated, are you willing to take the loss? Why??? If not, how will you deal with that? Re-negotiate the contract?
  • How do you know enough about the work to be able to give a resonably accurate price/estimate?
  • If a competitor shows them a better way, will they ever come back to you again?
  • That’s just a start – I’m sure we can come up with a lot of other things to consider.

If you choose to work following the “we must do estimates” model you are in good company (or at least in very numerous company), as there seems to be many who feel this is a reasonable way (or the only way they’ve found, perhaps?) to do business.

The Agile Way

I recently heard Jeff Sutherland do a talk where he mentioned the “80/20” rule – that “80 percent of the value is in 20 percent of the features” of a software project.  He quoted someone (I don’t remember who) as having suggested “we should just fund the 20% and not do the rest”.  I agree.  I’ll take it a bit further: I believe that the rule is more like “95/5” – that is: 95% of the value is in about 5% of the features, and that the people paying for the project will quickly end the project once that 5% is in use. Perhaps our goal is to pay as little as we can to discover and create that 5%.  So… how to do that?

Some choose the Agile way.  In this model, we recognize that “requirements emerge”.  If we can get good at allowing the more important and useful requirements to naturally emerge it pays off very nicely.

What does this have to do with estimates?  If I have to tell you, then you probably should be reading someone else’s blog.  But here it is: A customer who needs to “know how much it will cost” before they will decide to hire you to do the project might not be the sort who is willing to let requirements emerge.   It’s just not their way of thinking.  You might be able to “change them”, but probably not.

Not All Customers Require an Estimate (or price)

What it comes down to is this: You get to choose who you do business with.  If you choose to serve customers who need an estimate/price, then do estimates/prices, and figure out how to make that work for you.  If you choose to serve customers who are willing to let requirements emerge, then get good at the Agile way, and go for those customers.  It’s your choice.

Some of my other posts on estimates and “no estimates”

NOTES:

* For the purpose of this article, the sort of estimates I am discussing are the estimates typically asked for on many software development projects where a project, a feature, or a function, or a bug fix (or where a list of features or functions) are described and people are asked to come up with an approximate cost in time, money, or effort to do the work that will be required to provide the feature(s)/function(s)/capability(ies)/bug fix(es) being requested.

Disclaimer: There are many situations where estimates can be meaningful and useful.  This article is about situations where I don’t think they are typically meaningful or useful, and only in the realm of software development.

Agile2013 Lightning Talk: NoEstimates

NOTE: This was a lightning talk proposal for Agile 2013 in Nashville.  My main purpose in this post was to have a chance to give a lightning talk on “No Estimates” at that conference.  An additional purpose is to encourage people to be thinking about the sort of estimates we use in software development, why we think they are useful, why we depend on them so heavily, and some of the reasons we might want to re-think our dependence on estimates.

Fortunately for me, I was chosen to present this lightning talk, which challenged me to put together a focused slide deck and talk on the subject, which then grew into the talk which I gave at Oredev 2013.

Going forward, my is goal to expose the sort of thinking and questioning I have been doing over the last 10+ years so that others who have been asking can use these questions as a starting point in their own exploration of estimate-driven software development.

I am not offering to answer these questions for you.  Only you can do that.

I want to emphasize this: If you want to discuss these ideas with me, I am always happy to talk with almost anyone – either via Skype, or at a conference or developer meetup.  If you demand that I answer these questions for you, then you have misunderstood the purpose of this question list.  Again: I am not offering to answer these questions for you. I am offering these questions as a starting point for your own journey of exploration and learning.  I encourage you to use them.

Now: the original post:

I’m proposing an “idea” to present for the Agile2013 Lightning Talk stage under the Process At Scale theme.

Title: No Estimates

Talk summary: Ask a lot of questions, answer a few.

Please vote for it here: Agile2013 Lightning Talk On No Estimates

Here are the things I have been asking myself about Estimates:

  • What are estimates, anyways?
  • Are estimates useless? wasteful? harmful?
  • Who needs them? Who is asking for them?
  • Why do we think we need them?
  • Can we have successful projects without estimates?
  • When will this be done? How much will it cost?
  • How can we decide if we should do a project without knowing the cost?
  • Is NoEstimates better than MoEstimates?
  • How expensive are useful estimates?
  • How useful are inexpensive estimates?
  • If it takes you “no time” to do estimates, are they of any value?
  • If estimates are waste, would you still do them?
  • Is the reason you do estimates because “someone requires I do estimates”?
  • Why do they require you to do estimates – is it because someone “higher” is asking them for estimates?
  • When someone asks for an estimate, what do you do?
  • When estimates don’t work well for us, should we just get “better at doing estimates”?
  • If estimates are useful and easy to do, why do we still have so many problems with estimates?
  • How would you plan a project if you were NOT allowed to do estimates?
  • Can you imagine any reason why estimates might be harmful?
  • Can you imagine some better way?
  • How important are predictions?
  • Can you “sell” programming without estimates?
  • What about estimating the “value” of a feature?
  • Are there things where estimates make sense?
  • What is “No Estimates” anyway?
  • If estimates are good for some things, does that mean they are good for everything?
  • Does “No Estimates” mean we don’t use estimates at all?
  • What are “Whole Team Estimates” and does it help at all?
  • If you are estimating about work you don’t yet understand to be done by someone you haven’t yet hired, is that meaningful?
  • How much are you willing to invest in estimates to get useful information from them?
  • Does it matter how good estimates are: are “wild guesses” good enough?
  • Are the decisions you are making that are based on estimates good decisions?

We’ll ask some of these questions. Perhaps explore a few ideas. Maybe answer a few.

Stuff like that.

Why do we need estimates?

[NOTE: This post is about software development. You might need estimates in some other field or domain and this article is not related specifically to any other domain, and especially not the one you want to complain about.]

I’d like to do a little investigation about why we need estimates.

The following list contains some of the things people say when I ask “What do you use estimates* for in software development”:

  • To decide if we can do a project affordably enough to make a profit.
  • To decide what work we think we can do during “a Sprint”. (A decision about amount)
  • To decide what work we should try to do during “a Sprint”. (A decision about priority or importance)
  • To decide which is more valuable to us: this story or that story.
  • To decide what project we should do: Project A or Project B.
  • To decide how many developers/people to hire and how fast to “ramp up”.
  • To decide how much money we’ll need to staff a team for a year.
  • To give a price, or an approximate cost so a customer can decide to hire us to do their project.
  • So we can determine the team’s velocity.
  • So marketing can do whatever it is they do that requires they know 6 months in advance exactly what will be in our product.
  • Someone told me to make an estimate, I don’t use them for anything.

That’s enough for a start. So let’s start.

Let’s notice the most obvious common word in that grouping. To me, it is the word “decide”. Granted, the last few items don’t use the word “decide”, but we have to start somewhere. Besides, why would have I contrived that list and highlighted the word “decide” if it wasn’t fundamental to my main point? [At least I’m honest in stating that I have an agenda.  Hint: NO ESTIMATES]

It is significant to me that much of why we do estimates is that we need to make decisions: To decide something.

In other words, it looks to me that what we need are decisions, not estimates. Estimates are often used to help us make decisions. Estimates are not the only way, and for many of the decisions we need to make they are likely not the best way, in my opinion. We can,  and maybe we should explore other ways to make those same decisions. [Who are we kidding, I really think we need to find better ways to make those decisions].

So to restate: We don’t need estimates, we need to make decisions.

Most of us will probably agree that decisions are important and we need to make them, and that we want to make “good” decisions. Great decisions are even better. Yet in many cases, mediocre decisions are sufficient [stay with me here…]

Interestingly, as long as we pay attention, learn from our mistakes, and are willing to make “course corrections” frequently, even bad decisions are often sufficient. In other words, even if we make bad decisions, we can adjust and steer our work (and therefore our results) by making decisions as we go along.  That is an important point. Dwell on it.

Perhaps it would be advantageous to find ways to work where the quality of our decisions are not as important as our ability to learn and to adjust our course while paying as little as possible for the learning, adjusting, and “mistakes”.

To me, this is a “Lean approach” to making software. To me, this is the “Agile way”. Some people used to call this a “light-weight methodology” for developing software to indicate the process was “barely sufficient” rather than “cripplingly  comprehensive and depressingly dysfunctional”. I like “light-weight”, but someone picked the name “Agile” for this idea. I like that word even better. Regardless of the name, it’s an approach where we pay attention to what is going on and constantly adjust things based on what we learn.

One other thing about decisions

Sometimes decisions happen by default: If I forgot to bring my lunch, my options are to go hungry or eat cafeteria food today. If I forgot to bring money, then the “decision” was made for me – I go hungry. I prefer to not go hungry, but in some cases a default decision is sufficient. Make decisions on important things, resort to default decisions on less important things. Besides, I need to lose some weight anyway.

One other other thing about decisions

One little note: Of the items in the “What do you use estimates for in software development” list above, there is only one where estimates seem to make sense to me: To decide how much money we’ll need to staff a team for a year.  I’ll write about that someday if anyone is interested.  But I am confident you can figure it out yourself.  Also, there are only a couple of items in that list that seem like important decisions to me. But what do I know?

One other other other thing

As far as the “team velocity” question goes – you don’t need an estimate for that. To determine the velocity of a team simply drop them from a height of about 2000 feet and you can measure their speed just before they hit the ground.  I’ve never tried that, but I’ve worked some places where it felt like that.

In Conclusion… We need to make decisions, we don’t need estimates.  Did I say that already?

So the title of this post was “Why do we need estimates?”. The answer is, we need them only if we can’t find a better way to make decisions.

There is a lot of stuff we need to decide, which requires that we make decisions. Make many small decisions while you learn, steer, adjust, and correct.

Some other posts I’ve written on the topic of estimating:

Perhaps you are now saying: So what? I don’t know of a better way, so I am going to use estimates. Besides, what’s so bad about estimates?  Well… I’ve written a little about that (although I’ve really just started – there is a LOT of potentially bad stuff about estimates we should all be aware of), and about a few alternatives to using estimates. Over time, I’ll keep adding more information, and share more about how to make decisions without estimates. In the mean-time, I know that many others are also finding ways to make great decisions without estimates, and some of them are writing about it as well.  Google that.  You can probably figure this stuff out better on your own, anyway.

NOTES:

* For the purpose of this article, the sort of estimates I am discussing are the estimates typically asked for on many software development projects where a project, a feature, or a function, or a bug fix (or where a list of features or functions) are described and people are asked to come up with an approximate cost in time, money, or effort to do the work that will be required to provide the feature(s)/function(s)/capability(ies)/bug fix(es) being requested.

Disclaimer: There are many situations where estimates can be meaningful and useful.  This article is about situations where I don’t think they are typically meaningful or useful, and only in the realm of software development.

If You Found Estimates Bring No Value – What Would You Do?

I think this is a simple question:

In your software development efforts, would you eliminate estimates* if you found they bring no value? Put another way, if it costs (time, money, effort, whatever) to do estimates, and you get no return for that expense AT ALL, would you still do those estimates?

Few Are Willing To Answer This Question Outright

I’ve asked this question to about 50 people (face-to-face, in conference sessions, in “Skype” conversations, and in other forums) and here are some of the typical things I hear in response (note that these “answers” don’t answer the question at all):

  • I still like estimates, even if they bring no value, because they help me think about my work
  • We NEED estimates to make decisions
  • I just use them to determine velocity
  • My boss still requires them, even if I think they are waste
  • We need estimates to be able to bid jobs – our customers require them
  • It depends on the context

I rarely hear the answer I originally thought I would hear:

  • Dang right I’d eliminate them! Anything that is a cost that brings no value is WASTE and I would rather spend my money some better way. I won’t keep any process, practice, procedure, policy, or plan with no pay-off. (And quit spitting, will ya?)

I hear a more reasonable answer when I ask the question like this:

Same question (more or less) without the word “estimates”: If waste is anything that has a cost but has no value, would you eliminate waste from your process were you to find it?

Most of the time the answer I hear to this is “You bet I would”.

It seems to me, for the time being anyway, that there is some sort of problem people have when I insert “estimates” into the conversation.

The question is hypothetical

Why are few willing to actually answer the question, and instead “dodge it”?  I don’t know.  I might be asking it wrong.

I pose hypothetical questions to myself all the time.  One of my favorite “thinking tools” is to take the opposite side of some belief I have. For example: I love pair programming.  It feels right, helps me work faster and stay on track, accelerates my learning, gives me confidence in my code, things like that.  So, I will challenge myself and ask: What if I’m wrong? What if pair programming is a wasteful practice? How could I prove that? What would I do if I did  prove it?

Exploration of questions like these help me grow my ability to do continuous improvement. At least it feels that way to me.

I’m Dedicated To Making Things Better

I am always on the alert for waste. It comes in many forms, yet it is often hidden, or disguised as something of value. It’s fun, interesting, exciting even, to find and eliminate waste.  And, it is just plain good business.  A penny saved is ten cents earned.  If you don’t understand how that works, I’ll explain it some other time.

I find it helpful to remember the old Einstein saying about insanity, which I paraphrase here: We keep doing this, it doesn’t seem to work, we need to do better next time, we try to do better, things never get better. Perhaps “getting better at this” is failing, and “this” is not the right thing to do at all.

Once I’ve opened my eyes to the possibilities, I usually find big improvements are going to start happening.  The old thinking and old practices that have been blocking me from “better” are gone, and I can allow myself to imagine the possibilities, and to start innovating.

All I ask

So here is what I ask: Think of it like debate class.

The proposition: “Estimates are waste and we must eliminate estimates in software development”.

Take the affirmative position supporting the proposition, and prepare your arguments.  Suggest ways you could prove that estimates are a waste. Propose how you could replace estimates in your process.  Imagine you could convince your customer to hire your company without a bidding process – perhaps something more like choosing a doctor than buying a car.  Stuff like that.

We won’t really debate – this is just an exercise.  Are you willing to stretch your thinking a bit?  It shouldn’t take long. And you can quit any time and watch TV instead.

Some other posts I’ve written on the topic of estimating:

Cheers!

NOTES:

* For the purpose of this article, the sort of estimates I am discussing are the estimates typically asked for on many software development projects where a project, a feature, or a function, or a bug fix (or where a list of features or functions) are described and people are asked to come up with an approximate cost in time, money, or effort to do the work that will be required to provide the feature(s)/function(s)/capability(ies)/bug fix(es) being requested.

Disclaimer: There are many situations where estimates can be meaningful and useful.  This article is about situations where I don’t think they are typically meaningful or useful, and only in the realm of software development.

A Thing I Can Estimate

This is a post about a trivial thing that I can estimate, and there is a non-trivial point I want to make at the end.  So you can just skip-to-the-end if that is the sort of person you are.

I can estimate how long it will take me to drive to work.

I drive to work 5 days a week.   I need to get to my workplace, but I also want to pick up my daily oatmeal and drink for breakfast at a fast-food drive-thru.  Simple stuff.

Why do I need an estimate?

I need an estimate so I can be sure to leave home early enough to get to work on time.

I like to get to work 15 minutes early, and I’d like to make sure that I am never late except in exceptional cases.  So if I can estimate a time that will get me there about 15 minutes early, but never later than 5 minutes early that would work for me.  I don’t mind if I am 20 minutes early, but I don’t really want to be 25 minutes early.  So, I’m looking for a time range that gets me to work between 7:40 and 7:55, with the ideal spot being 7:45.

I want to be on time to work, with sufficient time to not worry about being late, and still not waste too much time just sitting there tweeting about estimates.

I drive pretty much the same route every day with about 10 minor alternatives available. I can take advantage of these alternatives (turn right here, or turn right a block up, stop at this fast-food or at that fast-food, etc) to make allowances for congestion at traffic lights, schools (school days are busy, holidays are not), drive-thru lines, and stuff like that – so I make a lot of little decisions on the way. For example: I’ll turn right at a light if there are a lot of cars backed up and take a chance that the next street isn’t backed up.  That allows me adjust to get a better chance at a good drive time.

I can estimate how long this will take, and use that estimate most of the time.

Yesterday it took 40 minutes, the day before 43 minutes, the day before that 36 minutes (school holiday).  Over the last year, I’ve found that it takes about 40 minutes to drive to work on a typical day.

If I leave at 7:05, I will almost always get to work in my target time-frame.

All well and good.  And that works great for me for typical days.

I can take advantage of daily changes

I “re-estimate” daily to adjust for school holidays, weather, etc. and vary the time I leave for work to compensate.

To clarify: I re-estimate every day to adjust for daily differences.  I can “re-estimate” quickly.  The cost is about 10 seconds of my time.

On a school holiday, I can sleep in 5 minutes. If it is raining, I must leave 10 minutes early.  If it is snowing, something is really wrong because it never snows here.

Why Does This Work?

You already know:

  • I have a lot of data – way more than is needed.
  • I know all the variables (weather/holiday/etc)
  • I know the work-arounds (alternative routes)
  • It’s essentially the same thing every day
  • There are almost no unknowns
  • There is almost nothing new to learn
  • I can trust the guy who is providing the numbers.  It’s me.

That is, it is trivial.

By the way –  I can’t control the variables to predict an exact time, so a range is sufficient.

Having the estimate is useful as it helps me optimize my sleep time and still reach work on time.

This is the Non Trivial Part: How Similar Is This To Computer Programming?

So, if it is useful to have an estimate, and we can make a meaningful estimate – then we have a good application of estimates.  That is the case with my “drive to work” example.

In my humble opinion, this is not similar to computer programming. We have the first part: We feel estimates would be useful, but we don’t have the second part – the estimates are not meaningful unless you are doing very trivial work.

With software development we don’t have much data, we don’t know the variables, we don’t know the work-arounds, each project is very different, there are lots of unknowns, there is much to learn, and I can’t trust anyone for “good numbers”, not even ME.  By the time we know enough about what we are doing to make useful estimates they are useless because we are DONE.

When we find our software development work is “easy to estimate” it is likely that it is trivial in nature.  That is, if it matches the nature of my “drive to work” example then I would question if that work is actually software development. In software development, when I find I am doing the same thing over and over, know all the variables, have few unknowns, have little to learn – then that’s the time I investigate ways to automate it and stop “programming” it.

Have fun.

And remember: This is ALL JUST MY OPINION.  Don’t do anything I say.  Don’t mention to anyone else that this makes sense to you, they will just say you are crazy.

 

Some other posts I’ve written on the topic of estimating:

Can We Code Without Estimates?

Can we write code, create software applications, without doing estimates*?  I think there are at least some cases where estimates are not needed.

In The Beginning

I bought a computer in 1982 or 83 for the purpose of managing aspects of a business I had.  My business was essentially a small custom manufacturing business of sorts, and I had very little money.  No capital, just cash flow. I would sell some work, get a deposit, buy the materials and supplies for the work, do the work, get the balance on delivery, pay some bills, pay the rent, and go on to the next job.  Every hour I had was needed to earn money,  or support the overhead of doing business.  Additionally, I would spend some time improving my capability to more effectively make money, reduce costs, or “take care of business”. Payback for that time needed to be quick, not in two weeks, but in two days, or two hours.  Every penny counted.

So I now had a computer, but none of the software I bought worked well enough to help me.  I wanted to do simple stuff – like mailing lists, simple bookkeeping, making estimates for manufacturing jobs, writing invoices –  stuff like that.  I also wanted to do some very specialized things, like controlling devices: material cutting plotters, measurement devices, and imaging equipment.  But software was expensive, and it was mostly useless in those days.

Computers can help us do bookkeeping, right?

One important reason I wanted a computer was to keep my books. I wanted to spend less time doing my bookkeeping.  I had bought an application that was supposed to do simple bookkeeping: Keep track of money spent, allocate it to varioius accounts, keep track of the checkbook, reconcile the checkbook, do income and expense reports,  profit and loss reports, year end stuff, and so on.  Trouble was – it didn’t work well at all.  It was difficult to use, made mistakes, and the data got corrupted after a couple months of using the application.  It was truly useless and worse: it wasted a lot of my time.  I tried several other applications that friends of mine were using – but none of them made things easier than doing the work by hand.  I wanted my computer and software to save time, and make it easier for me to focus on getting and doing work.  It wasn’t able to do that.

Earn as you Learn – Or At Least Try

So I decided to try writing my own “bookkeeping application”.  It’s just numbers, right?  Computers work with numbers, right?  Shouldn’t be too hard.  Before we go any further, I’ll “skip to the end” and let you know: It was not hard at all.  It turns out that computers ARE good at working with numbers, and you just have to find a reasonable way to let users interface with the computer, and some way to show stuff on a screen, and some way to send stuff to a printer, and some way to save data to a disk so you can use it the next time you want to work with it, and some way to do the calculations and other work actions.  These are all easy things.

I had no experience writing applications at that time beyond some games and little text editing apps and similar things that came from books I was learning from.  For my first project I needed something that would return back the time I spent right away.  Seemed the checkbook part of the bookkeeping app would be the place to start – you need to put expenses and income into the system before you can do much else, and it would save me some of the time it took to do that work by hand.

A Simple Checkbook Application – Find A Small Bit Of Potential Value

Balancing a checkbook is easy enough, it took me about 2 hours a week to do the hand entry of checks and deposits into the paper system my accountant had set up for me, and to update the balance.  It also took about 3 or 4 hours once a month to reconcile the checkbook.  I had always made mistakes in entry or calculation, so sometimes it would take longer.  I was spending about 11 to 14 hours (or more) each month doing the paperwork for my checkbook.  There was no Quicken in those days.

If I could write something that would allow me to enter a check into the system, enter a deposit into the system, reconcile the registry, catch mistakes, correct mistakes, and print out the details – that would save me 11 to 14 hours every month.  Well… not really: It would take some of my time to do the entry of checks and deposits, and to review the output, and correct mistakes.  So there is the value: A time savings of perhaps 8 hours a month.

I had NO IDEA how much time it would take to write that software.  I had nothing to base it on. It seemed worth a shot, but I had no way of knowing that.  The only sure things was that I’d learn a bit more about programming.  The key is: I had identified a small chunk of work with POTENTIAL value.

Learn By Doing, Decide Based on Real Time, Real Value

So, I had a vague idea about value.  I figured I could put in about an hour each night and I would soon learn if it was worth trying to write this code.  In doing the work I would learn the things I needed to learn to do the work, and I could make decisions about the value being delivered.  At any point I could decide to cancel the project or keep going.  The decision would be based on value delivered for time invested.  NOT value that MIGHT be delivered for time that MIGHT be needed.  That seems like a bad way to make decisions to me, at least in some cases. Real value, real time.  Take action, make decision, take action, make decision.  A small amount of real work delivers enough value upon which to make a decision.  Deliver early, then decide: Should we do more, or try again, or change directions, or expand or reduce… whatever.  Baby steps every step of the way.

The First Bit Of Functionality: Data Entry

The first value I could see: Ability to enter in checks (number, payee, date, amount, etc.), save as a file, and print it out on paper.  This would be exactly what I was currently doing with pen and paper, and I could do the totals by hand with a calculator just like I had been doing.  So that is where I started. Not much value, but provided a way to “amplify learning”.

Now, it seemed that this “functionality” could be “written” without writing any code.  The user (me) could just use the existing text editor to type in the data in a specific format.  How long did it take to code that?  No code, no time.  I just needed to define a format.  How robust was that?  Not very! It would be easy to make mistakes in data entry – but I needed to get real functionality quickly.  Same old same old.  That’s software development, so I’ve learned.  Once the code is in use the most painful stuff will be revealed, and we can fix that first.  Might never need to do anything more than the minimum.  Try to keep it that way.

Writing to a file was easy enough.  What to do next?

I wanted to calculate the balance of the checkbook. I would be very close to having something truly useful if I could do that. That would require that I take into consideration deposits as well. That was easy to do, of course. Deposits are positive, checks and withdrawals were negative.  Or something like that.

Again, I had only an hour a night or so to spend on this – so everything was very “bare bones”, but reading the file, reading the records, and accumulating a total was easy enough.  And essentially, the application was ready for production.  It was useful. Lame, but useful.  Value was delivered, and it showed the project was viable.  There were a number of features to add before it would be “easy to use”. I still often made data entry mistakes, but I could fix those by simply opening the file with the text editor and making the needed change to the data. Over time, I added much of that functionality, and eventually it became my bookkeeping application.  The development was paid for by the time I saved every month.

So, that was the start – No Estimates

I don’t think I did any estimates in this work.  If you think I did, please let me know and I’ll clarify the process.  Or maybe I was wrong!

The process for me: Choose something of potential value, find the simplest thing I can think of, and try to do it.  If it works, I go on to the next bit that seems to have value, if it doesn’t work I try something else. Repeat until the payback doesn’t seem worth it anymore – at which point I can move to another “project”.

This is a very minimal example. It’s real, and it is where I started.  I used this same basic pattern to write a lot of software for myself over the next 15 years.  I never had a need to estimate anything.  I didn’t even think of making estimates.   Someone once told me that necessity is the mother of invention.  I’m not sure about that, but I just haven’t had enough time to waste on inventing things that I don’t need.

I’ve used this same apporach on much larger projects, in various situations, and with a number of different projects types.  I think some form of this approach can work for just about any project.

Please don’t yell at me

Before you start yelling, and before the veins start popping out on your head, please understand: in this post I AM NOT SAYING THAT ESTIMATES ARE NOT NECESSARY FOR MANAGING SOFTWARE  DEVELOPMENT in your situation.  I am simply saying that estimates are not needed for designing, coding, testing, and delivering software into use.

The process you implement for managing the business of making software might require that you do estimates.   But that necessity has nothing to do with writing software – that necessity is imposed by people who haven’t yet invented a better way to manage their software development, their customer interactions, their collaborative abilities, etc.  Maybe you can find a better way.  Maybe there isn’t a better way.  Regardless, if there IS a better way, maybe you can take a tiny step every day that moves you to a better process that does not require that you do estimates.  That’s up to you.

Can We Code Without Estimates?

Still, I hope you can agree that we can write code without estimates.  If you don’t think we can, I’d like to hear about it – especially if I am wrong or am missing something.

Some other posts I’ve written on the topic of estimating:

NOTES:

* For the purpose of this article, the sort of estimates I am discussing are the estimates typically asked for on many software development projects where a project, a feature, or a function, or a bug fix (or where a list of features or functions) are described and people are asked to come up with an approximate cost in time, money, or effort to do the work that will be required to provide the feature(s)/function(s)/capability(ies)/bug fix(es) being requested.

Disclaimer: There are many situations where estimates can be meaningful and useful.  This article is about situations where I don’t think they are typically meaningful or useful, and only in the realm of software development.

 

A Comment And Response from Estimate Chess Post

Cory House ( blog  , twitter) wrote a comment on my Estimate Chess post, and I’d like to double purpose it by turning it into a post.

First – thanks for reading my post, Cory, and for making comments.  I appreciate that very much!!!

Cory’s Comment

I can appreciate the feeling of fruitlessness in some estimates. It’s hard to get right, particularly when requirements are weak. However, I believe a professional developer must be willing to take the risk and do the hard work of estimating when requested. Drawing a hard line of “no estimating” without considering context doesn’t come across professional or accommodating.

Your chess example is entertaining, but it’s also a great example of a place where I feel it’s unfair to expect an estimate. Given, it’s gray exactly how your story would translate to the development world, but it seems to describe a request to estimate a project without even vaguely reasonable requirements. I think we all agree that’s a waste of time.

If given sufficient requirements, we, as professionals, should be able to estimate our work. The world expects no less from engineers in other fields. Multiple contractors recently estimated the cost to finish my basement despite the fact that I had no floor plan, no idea where lights would be, only a vague idea on what word work I’d like, a bathroom of unknown size and quality, etc. If one of them refused to provide an estimate, I’d have gone elsewhere.

I land plenty of dev work on contract because I’ve gotten good at asking the right questions and sniffing out complexity so that I can provide reasonable estimates. Being skilled in estimation sets you apart whether you’re independent or full-time. Steve McConnell wrote an excellent book on the subject. http://www.amazon.com/Software-Estimation-Demystifying-Practices-Microsoft/dp/0735605351   That said, while I disagree with your answer to “What can we do?” as “Nothing.”, I enjoyed reading your blog. It’s fun reading and certainly food for thought.

My Response:

Hello Cory! Thanks for commenting on my post!

Well, as you can guess, I don’t agree with your take on things – but that’s why I have a blog, and you have a blog. We all get to write about things as we see them.

Regarding professionalism: I believe it is professional to find a better way to do all the things we do, and eliminate things that are wasteful. I see estimates and the estimating process as mostly waste, and as a professional I see no need to “take on the hard work of estimating when requested”. More directly: estimating is not hard to do, in my opinion. I’m not clear on why you might think it is. However, I also believe that estimates as used in software development typically (and almost always) result in misinformation that is used to make bad decisions. Which is why I follow an Agile approach to development. The Agile approach, at least as I do it, provides a lot of value and eliminates a lot of waste. And that is very professional. We’ll just have to agree to disagree on our thinking about professionalism and estimates.

I don’t think I made it clear in the post: The Chess analogy is to show that something as simple as Chess becomes complicated quickly. It is a VERY SIMPLE GAME, with very few variables, and yet it becomes too difficult to predict when we add a few simple constraints. And even without those constraints, there is little we can predict. Otherwise we wouldn’t even play the game, right? Remember: Don’t stretch an analogy too far – it will always break. With software projects, anything that is non-trivial is much more complicated than the simple chess example.

Cory: “If given sufficient requirements, we, as professionals, should be able to estimate our work”.

My Response: This is where we really diverge. I see estimates as waste, in the Lean sense of the word, and most other ways. Estimates are not part of what the customer buys. They don’t add value to the product. In a value stream map estimates are on the “non-value” track. They are merely ONE WAY people have attempted to be able to arrange working relationships and get software created. It makes sense, and it partly works – still, there is nothing about them that makes them necessary, or even beneficial to the actual creation of software. We can write software without them. We can do whole projects without them. And if we can find a way to make BETTER SOFTWARE at less cost, and to more quickly get the right software into use without estimates, wouldn’t it be professional and worthwhile to at least consider doing it that way? Hopefully we can agree on that.

Cory: “The world expects no less from engineers in other fields. Multiple contractors recently estimated the cost to finish my basement despite the fact that I had no floor plan, no idea where lights would be, only a vague idea on what word work I’d like, a bathroom of unknown size and quality, etc. If one of them refused to provide an estimate, I’d have gone elsewhere.”

My Response: Here we continue to think differently. There are countless examples in life where estimates work to some degree. We’ll discuss that some other day – most, if not all of them are NOT useful as models for software development. Here is where I’d like to leave it for the moment: If you could have had the work done on your basement for less money and get higher quality, and perhaps get it done even quicker – would you be willing to consider doing that? I know that there would be no way to know it was cheaper if you had no estimates – but conceptually: If you could get all the benefits I describe would you consider learning to do business that way? I hope we can agree that would be better all other things being equal.

We have contracts, and estimates, and predictions, and all the trappings of doing business this way because most people haven’t found a way to do things better – we fail each other, we don’t deliver as expected, we don’t trust each other, so we find ways to do business that gives us a chance to do business. So… we can try to get good at estimates, and it if serves you and your customers, more power to you.

Lastly:

Cory: “That said, while I disagree with your answer to “What can we do?” as “Nothing.”, I enjoyed reading your blog. It’s fun reading and certainly food for thought.”

My Response: I didn’t ask “What can we do?”. I asked “What can we do instead of doing estimates?”, and the answer was simply put as “Nothing”. My point being that we do not need to do anything “instead” – estimates are often simply UNNEEDED. And then I posed the more meaningful question: “How can we develop software without doing estimates?”. The nuances of this are important, and perhaps I was being too cryptic. At the time I wrote this post it seemed few were willing to explore these things. I hope I was wrong, and now I am certainly finding more people willing to entertain and even experiment with finding better ways. And some doing purely “no estimate” projects.

Overall, I have found the Agile way works well. When you have customers who see the benefits and find it works better for them than the old phased, predictive approach – then it is truly sweet.

Thanks for your kind words. I try to be fun, and I’m “all about” food for thought!!!

Cheers!

Notes From A Conference Session On No Estimates in Software Development

I’ve done 12+ sessions at Agile and Developer “user groups”, Agile or Dev conferences, college classes, and code camps where the main topic (or a big component) of the session was Estimates in Software Development.

Here are the notes from one such session I facilitated last year at an Open Space event (Agile Open California, North)

Session Title: Estimates – Can’t Live With Them, Can We Live Without Them?

The title was my own.  I’ve been talking about, and holding sessions on this subject for several years, and the previous time I was at this event a couple years earlier I did a session on “5 Whys on Why we Estimate”.  It was a very contentious session, with almost everyone (except me) feeling estimates are important and needed for software development, and the problem is just that we’re no good at it… so we just need to get better at doing estimates.  I don’t agree (as you probably know).

The “we just need to get better at it” argument sounds a lot like what I used to hear in “post-mortem” and “lessons learned” sessions in waterfall/phased project environments. A typical “take-away” from those sessions were things like this: “We missed our deadlines and had a lot of rework because the requirents kept changing, even though we have a change control process in place.  We need to get better at controlling changes to requirements”.  I hope you see that is very likely a bad conclusion.  Essentially, that line of thinking is that “since control isn’t working, we need more control.”

I’ve been suggesting that if we keep failing at getting good at estimates even though we are working hard at becoming better at doing so (and have been doing so for countless years), then maybe we are expecting estimates to do something for us that just can’t be done.   Maybe there are better ways.  [Just to be clear, for me – I feel I have found a much better way and it is Agile Software Developent as presented in the Agile Manifesto (Values and Principles) – the Agile MVP.]

My position: We need to get to the bottom of what it is we want from estimates and predictions, and find a better way to get to what we really want. [HINT: What we reallly want is Sucessful Software Development Efforts (or something like that, perhaps?) .  Or, as long as we are wishing, let’s just wish to win the Mega Lottery and then spend our day’s doing something we really enjoy. Like taking hikes or arguing – whatever you like.]

Participants:

There were initially 17 participants.  Developers, ScrumMasters, Agile Coaches, Product Managers, etc.  Almost everyone had some experience with Agile/Scrum/XP/Lean.  A few people joined in as the session proceeded, and a few others left.

Purpose:

Explore and discuss what we want out of estimates and the problems we’re having with estimates.

[NOTE: Everything will be tainted because Woody Zuill is facilitating this session, and he leans slightly toward the “No Estimates” approach.]

Process:

My approach for this session: Quickly explain my basic thoughts about the failure of estimates and the purpose of the session, do an information gathering exercise.  Ask why we feel we need estimates, have the participants write their thoughts on sticky notes, group them, and then discuss.

Information Gathering: Why do we “need” estimates

  • We gathered sticky notes to identify “why we ‘need’ estimates and put them on easel sheets
  • We then gathered the notes into groupings.
  • There were 2 main groupings: Estimates are for planning, and estimates are for manipulating
  • NOTE: Tne notes below were written by the participants.  Some are a bit cryptic. Not to worry, okay?

  • Here are the notes and the groupings:

Affinity Group: Estimates are for Planning

  • How many resources do we allocate to do the work
  • Length of engagement
  • Costs
  • To Predict Future Velocity
  • You need estimates to determine budget
  • To determine when we’ll finish
  • Target implementation time-frame & duration for resource allocation & cost estimates
  • When to commit to customer delivery
  • to decide whether or not to do a task
  • To Determine when a feature will be available
  • To determine when we’ll finish
  • To know when it is done
  • It drives the business
  • So we can prioritize
  • Business needs it for planning
  • Request funds for work
  • To help balance the work
  • Helps in managing resources
  • Determine if end result is worth the cost
  • To give the customer prediction – to meet customer expectations
  • CEO needs it.

Affinity Group: “Gaming” or manipulating the system and related stuff / dysfunctions

  • NOTE: I am not discussing these things in this post – sometime soon I’ll cover this stuff in detail.
  • The keep your job
  • To give a reason to yell at someone
  • It’s “How we do things”
  • Habit
  • Need to get a programmer fired (setting her up for failure)
  • To cover your own ass as a developer
  • To pit people/teams against each other

Now for the discussion

After our gathering and grouping exercise, we began discussing what we felt this showed us.  During the discussion we just “followed the flow” as ideas were expressed.

We observed, as a group, that estimates are mostly for planning and making decisions about what to work on.

In the discussion, we were in alignment that we “need” estimates so we can plan our work, and so we can make decisions about which projects or features to work on.

After all the effort we put into estimating, we still rarely get things done “on schedule” and “within budget”.

Also, we still frequently ask (or our managers ask): “Are we there yet?”.

Why do we have these problems?  We’ve been doing estimates for years – shouldn’t we be great at it by now? Other similar or related things that we’ve seen:

  • Progress isn’t visible
  • We seem to be get more behind our schedule no matter what we do
  • Money is running out and important things are not even close to being done.
  • Stuff we thought was done turns out to not be.

Rarely are the estimates accurate enough to really count on for release dates, costs, etc.

How do we typically deal with “over budget” and “not on schedule” issues?

  • We end up cutting features
  • Pushing out deadlines and so on
  • Hire more people
  • Blaming each other about whose “fault” it is
  • Agreeing to get “better” at gathering requirements and doing estimates “next time”

Still, others are dependent on estimates such as marketing, HR, Ops, our customers… and so on

  • Do estimates really provide sufficient information for these folks to get real value?
  • Or… are there a lot of decisions made based on faulty estimates???

So, With all these problems, why aren’t we looking for a better approach? What is wrong with our estimating process? Should we try to get better at it, or… ?”

One reason estimates provide little chance for usefulness:

  • The Unknowns might invalidate any estimate we make – estimates by nature contain unknowns
  • What is 4 x 8 + 7  / 2 * 83 + unknown?
    • The answer UNKNOWN
    • We can guess, or base our UNKNOWN on similar things.  Might work. Might not.
  • Many important decisions are based on UNKNOWN, but we act as if we “KNOW” something
  • The argument is often made that we just need to “have some relative estimate…”
    • It is likely that even this leads to bad decisions
    • For example, what good is relative decision based on “This UNKNOWN is bigger than that UNKNOWN?”

MY CONCLUSION AS PRESENTED:

So, if the Estimating process is flawed (and it might be), and getting “better at it” hasn’t worked… Do we really need estimates?  Can we innovate a better way?

  • What I am suggesting is that we have to evaluate our dependence on estimates
  • My own experiences have proven to me that estimates are not always needed.
    • I’ve been working without estimates for over 3 years with good results in corporate, “internal” projects
    • I’ve had several long term ongoing projects that did not use (or need) estimates – One of over 4 years on commercial software
    • I’ve done contract work with NO ESTIMATES. Many have told me “customers won’t do that”, but there are at least some that will.
      • When a customer sees that more “real” work gets done and more of the right thing gets done with less “busy work” they get on board.
      • Finding ways to pay a very low cost to prove something works, or doesn’t work is very attractive to most customers.  To me, this is a fundamental benefit of an Agile approach.
  • My experiences are not proof that estimates are not needed.
    • I am not trying to change the world.  I do want MY jobs/contracts/customers to get the benefit of doing things in a better way.
    • I AM trying to invite conversation about this stuff, and encourage us to explore and replace these painful practices in our profession.

This discussion had more to do with posing the questions than answering them

  • It is contextual (as usual)
  • Lots of organizations feel they need estimates and you must work within the system.  Sorry for that – but I can’t fix that.
  • Often, customers will not entertain using a company that does not provide an estimate.
    • They want to know what they’ll get, when they’ll get it, and how much it will cost.
    • It is a risky situation for everyone – and if you are in that world, there are still ways “sell” the benefits of finding a better way.
  • I am suggesting is that we need to do some serious thinking about how to change this reality, and innovate ways to eliminate the dependence on estimates

I am proposing that we need to imagine a different world

  • Do some “thought” experiments
    • IMAGINE:
      • What if we were to find that estimates have NO value (zero benefit), should we still do them?
        • What would that look like?
      • What if we were to find that estimates are harmful and are counter-productive?
        • What would you do?
      • What if the were to find out that estimates are fatally TOXIC
        • What would we do then?  How fast would we need to address this?

Take some action that will expose the reality about our estimates

  • Can we find a way to wean ourselves from estimate dependency?
  • A few things that have worked for me
    • Deliver value quickly and frequently (sound familiar?) – that is: put small chunks into real use early and often
      • Making decisions about “what to do” become less important when we can do “some of this” then “some of that” and see what works (or doesn’t work) quickly and cheaply.
      • People stop worrying about “are we there yet” when interesting (useful) things are continuously happening and being put into real use by real users.
        • It’s like bringing along some new toys and activities to hand to your kids on long trips when they start getting antsy.
    • Lead your customer/stakeholders/whatever to an understanding of what is important to deliver NEXT…
      • … we only have so much work that can be done right now – focus on that
      • … put all your effort into that
      • … and put little time on all else
    • You can introduce this a little at a time.
      • Thinking people who are open to this will slowly come around.  Or not.  Just depends.

Remember: I am just encouraging discussion.

What works for me might not make sense to you.  Maybe someday it will.  When that happens, you will have crossed over into the land of no estimates.  Beware.