What do I know?
Archive for the ‘Estimating’ Category.
I was invited to present my ideas about “No Estimates” at Oredev 2013 in Malmo, Sweden. It was an exciting and wonderful experience for me (to say the least), and perhaps someday soon I’ll post more about it.
Until then, here is a link to a video of the session I presented there about #NoEstimates:
Please let me know what you think, if you would.
In Twitter, we use the convention of “hashtags” to connect a tweet with a theme or topic so that it is easy (or at least a bit easier) to search and find tweets for topics we want to follow.
Several people have blamed me for coining the “#NoEstimates” hashtag. It might be true – I’m not sure. It is possible that I was the first to use this hashtag for Software Development – a search of Twitter history on it shows that, as far as I can tell.
[UPDATE: I found the following post from Aslak Hellesøy – @aslak_hellesoy dated Feb 10, 2010 that used this hashtag
@obie at #speakerconf: “Velocity is important for budgeting”. Disagree. Measuring cycle time is a richer metric. #kanban #noestimates.]
Here it is, as defined by me. Others probably have their own idea:
#NoEstimates is a hashtag for the topic of exploring alternatives to estimates [of time, effort, cost] for making decisions in software development. That is, ways to make decisions with “No Estimates”.
I’ll probably modify this a bit over time – I put this together quickly as a starting point.
Estimates are typically and pervasively used in the decision-making process throughout software development organizations and efforts – both in “phased/waterfall” and “Agile” methodologies. They are ubiquitous – it is rare to find a project/organization/effort where it isn’t simply accepted that “we must have estimates – it is the only way”.
Additionally, for a long time I’ve been noticing that almost every training, book, conference session, article, or blog on “Agile” software development has a very heavy focus on estimating. I’m fine with important things taking a lot of our attention – they are important, after all. Are estimates that important?
I’ve also noticed over the years that many organizations have a number of dysfunctions in how estimates are handled, and the overall decision making process that depends on estimates. Have you noticed any dysfunctions?
So… I started doing “5 why sessions” or similar exercises at conferences and user groups, and other investigations into the reasons for this, and the dysfunctions that people have been experiencing. I also started discussing alternative ways to manage software development efforts without using estimates. I’ve been working without estimates for years for many decisions that are typically “estimate-driven” in software development management, and some people are interested in how to do this.
It’s important to ask ourselves questions such as: Do we really need estimates? Are they really that important? Are there options? Are there other ways to do things? Are there BETTER ways to do thing?
If we are willing to simply play the “devil’s advocate”, and imagine (or at least temporarily pretend) that perhaps there are better ways to make decisions, it can lead to more open thinking and clarity as to why we “believe the things we believe”. Why not question the things we hold so dearly?
It seems a lot of people have an argument against the “No Estimates” idea that goes something like this:
There are many variations on this, but they typically come down to the same thing: We need customers, customers need estimates, therefore, we do estimates. The following are a few generic examples of this idea:
This is a very easy question to answer: You probably can’t get these customers without estimates. They want estimates – it’s part of the way they think. You probably MUST DO ESTIMATES for these customers. That is, you don’t tell these customers “We don’t do estimates” if you want to work with them.
If you have chosen to work with the sort of customer that requires an estimate (or price) then give them what they want: an estimate (or price). Your main job is to figure out a way to be able to make money taking on work for this sort of customer. I understand that. You understand that. What more do you need to know?
If you choose to work following the “we must do estimates” model you are in good company (or at least in very numerous company), as there seems to be many who feel this is a reasonable way (or the only way they’ve found, perhaps?) to do business.
I recently heard Jeff Sutherland do a talk where he mentioned the “80/20” rule – that “80 percent of the value is in 20 percent of the features” of a software project. He quoted someone (I don’t remember who) as having suggested “we should just fund the 20% and not do the rest”. I agree. I’ll take it a bit further: I believe that the rule is more like “95/5” – that is: 95% of the value is in about 5% of the features, and that the people paying for the project will quickly end the project once that 5% is in use. Perhaps our goal is to pay as little as we can to discover and create that 5%. So… how to do that?
Some choose the Agile way. In this model, we recognize that “requirements emerge”. If we can get good at allowing the more important and useful requirements to naturally emerge it pays off very nicely.
What does this have to do with estimates? If I have to tell you, then you probably should be reading someone else’s blog. But here it is: A customer who needs to “know how much it will cost” before they will decide to hire you to do the project might not be the sort who is willing to let requirements emerge. It’s just not their way of thinking. You might be able to “change them”, but probably not.
What it comes down to is this: You get to choose who you do business with. If you choose to serve customers who need an estimate/price, then do estimates/prices, and figure out how to make that work for you. If you choose to serve customers who are willing to let requirements emerge, then get good at the Agile way, and go for those customers. It’s your choice.
NOTES:
* For the purpose of this article, the sort of estimates I am discussing are the estimates typically asked for on many software development projects where a project, a feature, or a function, or a bug fix (or where a list of features or functions) are described and people are asked to come up with an approximate cost in time, money, or effort to do the work that will be required to provide the feature(s)/function(s)/capability(ies)/bug fix(es) being requested.
Disclaimer: There are many situations where estimates can be meaningful and useful. This article is about situations where I don’t think they are typically meaningful or useful, and only in the realm of software development.
NOTE: This was a lightning talk proposal for Agile 2013 in Nashville. My main purpose in this post was to have a chance to give a lightning talk on “No Estimates” at that conference. An additional purpose is to encourage people to be thinking about the sort of estimates we use in software development, why we think they are useful, why we depend on them so heavily, and some of the reasons we might want to re-think our dependence on estimates.
Fortunately for me, I was chosen to present this lightning talk, which challenged me to put together a focused slide deck and talk on the subject, which then grew into the talk which I gave at Oredev 2013.
Going forward, my is goal to expose the sort of thinking and questioning I have been doing over the last 10+ years so that others who have been asking can use these questions as a starting point in their own exploration of estimate-driven software development.
I am not offering to answer these questions for you. Only you can do that.
I want to emphasize this: If you want to discuss these ideas with me, I am always happy to talk with almost anyone – either via Skype, or at a conference or developer meetup. If you demand that I answer these questions for you, then you have misunderstood the purpose of this question list. Again: I am not offering to answer these questions for you. I am offering these questions as a starting point for your own journey of exploration and learning. I encourage you to use them.
Now: the original post:
Please vote for it here: Agile2013 Lightning Talk On No Estimates
Here are the things I have been asking myself about Estimates:
We’ll ask some of these questions. Perhaps explore a few ideas. Maybe answer a few.
Stuff like that.
[NOTE: This post is about software development. You might need estimates in some other field or domain and this article is not related specifically to any other domain, and especially not the one you want to complain about.]
I’d like to do a little investigation about why we need estimates.
The following list contains some of the things people say when I ask “What do you use estimates* for in software development”:
Let’s notice the most obvious common word in that grouping. To me, it is the word “decide”. Granted, the last few items don’t use the word “decide”, but we have to start somewhere. Besides, why would have I contrived that list and highlighted the word “decide” if it wasn’t fundamental to my main point? [At least I’m honest in stating that I have an agenda. Hint: NO ESTIMATES]
It is significant to me that much of why we do estimates is that we need to make decisions: To decide something.
In other words, it looks to me that what we need are decisions, not estimates. Estimates are often used to help us make decisions. Estimates are not the only way, and for many of the decisions we need to make they are likely not the best way, in my opinion. We can, and maybe we should explore other ways to make those same decisions. [Who are we kidding, I really think we need to find better ways to make those decisions].
Most of us will probably agree that decisions are important and we need to make them, and that we want to make “good” decisions. Great decisions are even better. Yet in many cases, mediocre decisions are sufficient [stay with me here…]
Interestingly, as long as we pay attention, learn from our mistakes, and are willing to make “course corrections” frequently, even bad decisions are often sufficient. In other words, even if we make bad decisions, we can adjust and steer our work (and therefore our results) by making decisions as we go along. That is an important point. Dwell on it.
Perhaps it would be advantageous to find ways to work where the quality of our decisions are not as important as our ability to learn and to adjust our course while paying as little as possible for the learning, adjusting, and “mistakes”.
To me, this is a “Lean approach” to making software. To me, this is the “Agile way”. Some people used to call this a “light-weight methodology” for developing software to indicate the process was “barely sufficient” rather than “cripplingly comprehensive and depressingly dysfunctional”. I like “light-weight”, but someone picked the name “Agile” for this idea. I like that word even better. Regardless of the name, it’s an approach where we pay attention to what is going on and constantly adjust things based on what we learn.
Sometimes decisions happen by default: If I forgot to bring my lunch, my options are to go hungry or eat cafeteria food today. If I forgot to bring money, then the “decision” was made for me – I go hungry. I prefer to not go hungry, but in some cases a default decision is sufficient. Make decisions on important things, resort to default decisions on less important things. Besides, I need to lose some weight anyway.
One little note: Of the items in the “What do you use estimates for in software development” list above, there is only one where estimates seem to make sense to me: To decide how much money we’ll need to staff a team for a year. I’ll write about that someday if anyone is interested. But I am confident you can figure it out yourself. Also, there are only a couple of items in that list that seem like important decisions to me. But what do I know?
As far as the “team velocity” question goes – you don’t need an estimate for that. To determine the velocity of a team simply drop them from a height of about 2000 feet and you can measure their speed just before they hit the ground. I’ve never tried that, but I’ve worked some places where it felt like that.
So the title of this post was “Why do we need estimates?”. The answer is, we need them only if we can’t find a better way to make decisions.
There is a lot of stuff we need to decide, which requires that we make decisions. Make many small decisions while you learn, steer, adjust, and correct.
Perhaps you are now saying: So what? I don’t know of a better way, so I am going to use estimates. Besides, what’s so bad about estimates? Well… I’ve written a little about that (although I’ve really just started – there is a LOT of potentially bad stuff about estimates we should all be aware of), and about a few alternatives to using estimates. Over time, I’ll keep adding more information, and share more about how to make decisions without estimates. In the mean-time, I know that many others are also finding ways to make great decisions without estimates, and some of them are writing about it as well. Google that. You can probably figure this stuff out better on your own, anyway.
NOTES:
* For the purpose of this article, the sort of estimates I am discussing are the estimates typically asked for on many software development projects where a project, a feature, or a function, or a bug fix (or where a list of features or functions) are described and people are asked to come up with an approximate cost in time, money, or effort to do the work that will be required to provide the feature(s)/function(s)/capability(ies)/bug fix(es) being requested.
Disclaimer: There are many situations where estimates can be meaningful and useful. This article is about situations where I don’t think they are typically meaningful or useful, and only in the realm of software development.
In your software development efforts, would you eliminate estimates* if you found they bring no value? Put another way, if it costs (time, money, effort, whatever) to do estimates, and you get no return for that expense AT ALL, would you still do those estimates?
I’ve asked this question to about 50 people (face-to-face, in conference sessions, in “Skype” conversations, and in other forums) and here are some of the typical things I hear in response (note that these “answers” don’t answer the question at all):
I rarely hear the answer I originally thought I would hear:
Same question (more or less) without the word “estimates”: If waste is anything that has a cost but has no value, would you eliminate waste from your process were you to find it?
Most of the time the answer I hear to this is “You bet I would”.
It seems to me, for the time being anyway, that there is some sort of problem people have when I insert “estimates” into the conversation.
Why are few willing to actually answer the question, and instead “dodge it”? I don’t know. I might be asking it wrong.
I pose hypothetical questions to myself all the time. One of my favorite “thinking tools” is to take the opposite side of some belief I have. For example: I love pair programming. It feels right, helps me work faster and stay on track, accelerates my learning, gives me confidence in my code, things like that. So, I will challenge myself and ask: What if I’m wrong? What if pair programming is a wasteful practice? How could I prove that? What would I do if I did prove it?
Exploration of questions like these help me grow my ability to do continuous improvement. At least it feels that way to me.
I am always on the alert for waste. It comes in many forms, yet it is often hidden, or disguised as something of value. It’s fun, interesting, exciting even, to find and eliminate waste. And, it is just plain good business. A penny saved is ten cents earned. If you don’t understand how that works, I’ll explain it some other time.
I find it helpful to remember the old Einstein saying about insanity, which I paraphrase here: We keep doing this, it doesn’t seem to work, we need to do better next time, we try to do better, things never get better. Perhaps “getting better at this” is failing, and “this” is not the right thing to do at all.
Once I’ve opened my eyes to the possibilities, I usually find big improvements are going to start happening. The old thinking and old practices that have been blocking me from “better” are gone, and I can allow myself to imagine the possibilities, and to start innovating.
So here is what I ask: Think of it like debate class.
The proposition: “Estimates are waste and we must eliminate estimates in software development”.
Take the affirmative position supporting the proposition, and prepare your arguments. Suggest ways you could prove that estimates are a waste. Propose how you could replace estimates in your process. Imagine you could convince your customer to hire your company without a bidding process – perhaps something more like choosing a doctor than buying a car. Stuff like that.
We won’t really debate – this is just an exercise. Are you willing to stretch your thinking a bit? It shouldn’t take long. And you can quit any time and watch TV instead.
NOTES:
* For the purpose of this article, the sort of estimates I am discussing are the estimates typically asked for on many software development projects where a project, a feature, or a function, or a bug fix (or where a list of features or functions) are described and people are asked to come up with an approximate cost in time, money, or effort to do the work that will be required to provide the feature(s)/function(s)/capability(ies)/bug fix(es) being requested.
Disclaimer: There are many situations where estimates can be meaningful and useful. This article is about situations where I don’t think they are typically meaningful or useful, and only in the realm of software development.
This is a post about a trivial thing that I can estimate, and there is a non-trivial point I want to make at the end. So you can just skip-to-the-end if that is the sort of person you are.
I drive to work 5 days a week. I need to get to my workplace, but I also want to pick up my daily oatmeal and drink for breakfast at a fast-food drive-thru. Simple stuff.
I need an estimate so I can be sure to leave home early enough to get to work on time.
I like to get to work 15 minutes early, and I’d like to make sure that I am never late except in exceptional cases. So if I can estimate a time that will get me there about 15 minutes early, but never later than 5 minutes early that would work for me. I don’t mind if I am 20 minutes early, but I don’t really want to be 25 minutes early. So, I’m looking for a time range that gets me to work between 7:40 and 7:55, with the ideal spot being 7:45.
I want to be on time to work, with sufficient time to not worry about being late, and still not waste too much time just sitting there tweeting about estimates.
I drive pretty much the same route every day with about 10 minor alternatives available. I can take advantage of these alternatives (turn right here, or turn right a block up, stop at this fast-food or at that fast-food, etc) to make allowances for congestion at traffic lights, schools (school days are busy, holidays are not), drive-thru lines, and stuff like that – so I make a lot of little decisions on the way. For example: I’ll turn right at a light if there are a lot of cars backed up and take a chance that the next street isn’t backed up. That allows me adjust to get a better chance at a good drive time.
Yesterday it took 40 minutes, the day before 43 minutes, the day before that 36 minutes (school holiday). Over the last year, I’ve found that it takes about 40 minutes to drive to work on a typical day.
If I leave at 7:05, I will almost always get to work in my target time-frame.
All well and good. And that works great for me for typical days.
I “re-estimate” daily to adjust for school holidays, weather, etc. and vary the time I leave for work to compensate.
To clarify: I re-estimate every day to adjust for daily differences. I can “re-estimate” quickly. The cost is about 10 seconds of my time.
On a school holiday, I can sleep in 5 minutes. If it is raining, I must leave 10 minutes early. If it is snowing, something is really wrong because it never snows here.
You already know:
That is, it is trivial.
By the way – I can’t control the variables to predict an exact time, so a range is sufficient.
Having the estimate is useful as it helps me optimize my sleep time and still reach work on time.
So, if it is useful to have an estimate, and we can make a meaningful estimate – then we have a good application of estimates. That is the case with my “drive to work” example.
In my humble opinion, this is not similar to computer programming. We have the first part: We feel estimates would be useful, but we don’t have the second part – the estimates are not meaningful unless you are doing very trivial work.
With software development we don’t have much data, we don’t know the variables, we don’t know the work-arounds, each project is very different, there are lots of unknowns, there is much to learn, and I can’t trust anyone for “good numbers”, not even ME. By the time we know enough about what we are doing to make useful estimates they are useless because we are DONE.
When we find our software development work is “easy to estimate” it is likely that it is trivial in nature. That is, if it matches the nature of my “drive to work” example then I would question if that work is actually software development. In software development, when I find I am doing the same thing over and over, know all the variables, have few unknowns, have little to learn – then that’s the time I investigate ways to automate it and stop “programming” it.
And remember: This is ALL JUST MY OPINION. Don’t do anything I say. Don’t mention to anyone else that this makes sense to you, they will just say you are crazy.
Can we write code, create software applications, without doing estimates*? I think there are at least some cases where estimates are not needed.
I bought a computer in 1982 or 83 for the purpose of managing aspects of a business I had. My business was essentially a small custom manufacturing business of sorts, and I had very little money. No capital, just cash flow. I would sell some work, get a deposit, buy the materials and supplies for the work, do the work, get the balance on delivery, pay some bills, pay the rent, and go on to the next job. Every hour I had was needed to earn money, or support the overhead of doing business. Additionally, I would spend some time improving my capability to more effectively make money, reduce costs, or “take care of business”. Payback for that time needed to be quick, not in two weeks, but in two days, or two hours. Every penny counted.
So I now had a computer, but none of the software I bought worked well enough to help me. I wanted to do simple stuff – like mailing lists, simple bookkeeping, making estimates for manufacturing jobs, writing invoices – stuff like that. I also wanted to do some very specialized things, like controlling devices: material cutting plotters, measurement devices, and imaging equipment. But software was expensive, and it was mostly useless in those days.
One important reason I wanted a computer was to keep my books. I wanted to spend less time doing my bookkeeping. I had bought an application that was supposed to do simple bookkeeping: Keep track of money spent, allocate it to varioius accounts, keep track of the checkbook, reconcile the checkbook, do income and expense reports, profit and loss reports, year end stuff, and so on. Trouble was – it didn’t work well at all. It was difficult to use, made mistakes, and the data got corrupted after a couple months of using the application. It was truly useless and worse: it wasted a lot of my time. I tried several other applications that friends of mine were using – but none of them made things easier than doing the work by hand. I wanted my computer and software to save time, and make it easier for me to focus on getting and doing work. It wasn’t able to do that.
So I decided to try writing my own “bookkeeping application”. It’s just numbers, right? Computers work with numbers, right? Shouldn’t be too hard. Before we go any further, I’ll “skip to the end” and let you know: It was not hard at all. It turns out that computers ARE good at working with numbers, and you just have to find a reasonable way to let users interface with the computer, and some way to show stuff on a screen, and some way to send stuff to a printer, and some way to save data to a disk so you can use it the next time you want to work with it, and some way to do the calculations and other work actions. These are all easy things.
I had no experience writing applications at that time beyond some games and little text editing apps and similar things that came from books I was learning from. For my first project I needed something that would return back the time I spent right away. Seemed the checkbook part of the bookkeeping app would be the place to start – you need to put expenses and income into the system before you can do much else, and it would save me some of the time it took to do that work by hand.
Balancing a checkbook is easy enough, it took me about 2 hours a week to do the hand entry of checks and deposits into the paper system my accountant had set up for me, and to update the balance. It also took about 3 or 4 hours once a month to reconcile the checkbook. I had always made mistakes in entry or calculation, so sometimes it would take longer. I was spending about 11 to 14 hours (or more) each month doing the paperwork for my checkbook. There was no Quicken in those days.
If I could write something that would allow me to enter a check into the system, enter a deposit into the system, reconcile the registry, catch mistakes, correct mistakes, and print out the details – that would save me 11 to 14 hours every month. Well… not really: It would take some of my time to do the entry of checks and deposits, and to review the output, and correct mistakes. So there is the value: A time savings of perhaps 8 hours a month.
I had NO IDEA how much time it would take to write that software. I had nothing to base it on. It seemed worth a shot, but I had no way of knowing that. The only sure things was that I’d learn a bit more about programming. The key is: I had identified a small chunk of work with POTENTIAL value.
So, I had a vague idea about value. I figured I could put in about an hour each night and I would soon learn if it was worth trying to write this code. In doing the work I would learn the things I needed to learn to do the work, and I could make decisions about the value being delivered. At any point I could decide to cancel the project or keep going. The decision would be based on value delivered for time invested. NOT value that MIGHT be delivered for time that MIGHT be needed. That seems like a bad way to make decisions to me, at least in some cases. Real value, real time. Take action, make decision, take action, make decision. A small amount of real work delivers enough value upon which to make a decision. Deliver early, then decide: Should we do more, or try again, or change directions, or expand or reduce… whatever. Baby steps every step of the way.
The first value I could see: Ability to enter in checks (number, payee, date, amount, etc.), save as a file, and print it out on paper. This would be exactly what I was currently doing with pen and paper, and I could do the totals by hand with a calculator just like I had been doing. So that is where I started. Not much value, but provided a way to “amplify learning”.
Now, it seemed that this “functionality” could be “written” without writing any code. The user (me) could just use the existing text editor to type in the data in a specific format. How long did it take to code that? No code, no time. I just needed to define a format. How robust was that? Not very! It would be easy to make mistakes in data entry – but I needed to get real functionality quickly. Same old same old. That’s software development, so I’ve learned. Once the code is in use the most painful stuff will be revealed, and we can fix that first. Might never need to do anything more than the minimum. Try to keep it that way.
I wanted to calculate the balance of the checkbook. I would be very close to having something truly useful if I could do that. That would require that I take into consideration deposits as well. That was easy to do, of course. Deposits are positive, checks and withdrawals were negative. Or something like that.
Again, I had only an hour a night or so to spend on this – so everything was very “bare bones”, but reading the file, reading the records, and accumulating a total was easy enough. And essentially, the application was ready for production. It was useful. Lame, but useful. Value was delivered, and it showed the project was viable. There were a number of features to add before it would be “easy to use”. I still often made data entry mistakes, but I could fix those by simply opening the file with the text editor and making the needed change to the data. Over time, I added much of that functionality, and eventually it became my bookkeeping application. The development was paid for by the time I saved every month.
I don’t think I did any estimates in this work. If you think I did, please let me know and I’ll clarify the process. Or maybe I was wrong!
The process for me: Choose something of potential value, find the simplest thing I can think of, and try to do it. If it works, I go on to the next bit that seems to have value, if it doesn’t work I try something else. Repeat until the payback doesn’t seem worth it anymore – at which point I can move to another “project”.
This is a very minimal example. It’s real, and it is where I started. I used this same basic pattern to write a lot of software for myself over the next 15 years. I never had a need to estimate anything. I didn’t even think of making estimates. Someone once told me that necessity is the mother of invention. I’m not sure about that, but I just haven’t had enough time to waste on inventing things that I don’t need.
I’ve used this same apporach on much larger projects, in various situations, and with a number of different projects types. I think some form of this approach can work for just about any project.
Before you start yelling, and before the veins start popping out on your head, please understand: in this post I AM NOT SAYING THAT ESTIMATES ARE NOT NECESSARY FOR MANAGING SOFTWARE DEVELOPMENT in your situation. I am simply saying that estimates are not needed for designing, coding, testing, and delivering software into use.
The process you implement for managing the business of making software might require that you do estimates. But that necessity has nothing to do with writing software – that necessity is imposed by people who haven’t yet invented a better way to manage their software development, their customer interactions, their collaborative abilities, etc. Maybe you can find a better way. Maybe there isn’t a better way. Regardless, if there IS a better way, maybe you can take a tiny step every day that moves you to a better process that does not require that you do estimates. That’s up to you.
Still, I hope you can agree that we can write code without estimates. If you don’t think we can, I’d like to hear about it – especially if I am wrong or am missing something.
NOTES:
* For the purpose of this article, the sort of estimates I am discussing are the estimates typically asked for on many software development projects where a project, a feature, or a function, or a bug fix (or where a list of features or functions) are described and people are asked to come up with an approximate cost in time, money, or effort to do the work that will be required to provide the feature(s)/function(s)/capability(ies)/bug fix(es) being requested.
Disclaimer: There are many situations where estimates can be meaningful and useful. This article is about situations where I don’t think they are typically meaningful or useful, and only in the realm of software development.
Cory House ( blog , twitter) wrote a comment on my Estimate Chess post, and I’d like to double purpose it by turning it into a post.
First – thanks for reading my post, Cory, and for making comments. I appreciate that very much!!!
I can appreciate the feeling of fruitlessness in some estimates. It’s hard to get right, particularly when requirements are weak. However, I believe a professional developer must be willing to take the risk and do the hard work of estimating when requested. Drawing a hard line of “no estimating” without considering context doesn’t come across professional or accommodating.
Your chess example is entertaining, but it’s also a great example of a place where I feel it’s unfair to expect an estimate. Given, it’s gray exactly how your story would translate to the development world, but it seems to describe a request to estimate a project without even vaguely reasonable requirements. I think we all agree that’s a waste of time.
If given sufficient requirements, we, as professionals, should be able to estimate our work. The world expects no less from engineers in other fields. Multiple contractors recently estimated the cost to finish my basement despite the fact that I had no floor plan, no idea where lights would be, only a vague idea on what word work I’d like, a bathroom of unknown size and quality, etc. If one of them refused to provide an estimate, I’d have gone elsewhere.
I land plenty of dev work on contract because I’ve gotten good at asking the right questions and sniffing out complexity so that I can provide reasonable estimates. Being skilled in estimation sets you apart whether you’re independent or full-time. Steve McConnell wrote an excellent book on the subject. http://www.amazon.com/Software-Estimation-Demystifying-Practices-Microsoft/dp/0735605351 That said, while I disagree with your answer to “What can we do?” as “Nothing.”, I enjoyed reading your blog. It’s fun reading and certainly food for thought.
Hello Cory! Thanks for commenting on my post!
Well, as you can guess, I don’t agree with your take on things – but that’s why I have a blog, and you have a blog. We all get to write about things as we see them.
Regarding professionalism: I believe it is professional to find a better way to do all the things we do, and eliminate things that are wasteful. I see estimates and the estimating process as mostly waste, and as a professional I see no need to “take on the hard work of estimating when requested”. More directly: estimating is not hard to do, in my opinion. I’m not clear on why you might think it is. However, I also believe that estimates as used in software development typically (and almost always) result in misinformation that is used to make bad decisions. Which is why I follow an Agile approach to development. The Agile approach, at least as I do it, provides a lot of value and eliminates a lot of waste. And that is very professional. We’ll just have to agree to disagree on our thinking about professionalism and estimates.
I don’t think I made it clear in the post: The Chess analogy is to show that something as simple as Chess becomes complicated quickly. It is a VERY SIMPLE GAME, with very few variables, and yet it becomes too difficult to predict when we add a few simple constraints. And even without those constraints, there is little we can predict. Otherwise we wouldn’t even play the game, right? Remember: Don’t stretch an analogy too far – it will always break. With software projects, anything that is non-trivial is much more complicated than the simple chess example.
Cory: “If given sufficient requirements, we, as professionals, should be able to estimate our work”.
My Response: This is where we really diverge. I see estimates as waste, in the Lean sense of the word, and most other ways. Estimates are not part of what the customer buys. They don’t add value to the product. In a value stream map estimates are on the “non-value” track. They are merely ONE WAY people have attempted to be able to arrange working relationships and get software created. It makes sense, and it partly works – still, there is nothing about them that makes them necessary, or even beneficial to the actual creation of software. We can write software without them. We can do whole projects without them. And if we can find a way to make BETTER SOFTWARE at less cost, and to more quickly get the right software into use without estimates, wouldn’t it be professional and worthwhile to at least consider doing it that way? Hopefully we can agree on that.
Cory: “The world expects no less from engineers in other fields. Multiple contractors recently estimated the cost to finish my basement despite the fact that I had no floor plan, no idea where lights would be, only a vague idea on what word work I’d like, a bathroom of unknown size and quality, etc. If one of them refused to provide an estimate, I’d have gone elsewhere.”
My Response: Here we continue to think differently. There are countless examples in life where estimates work to some degree. We’ll discuss that some other day – most, if not all of them are NOT useful as models for software development. Here is where I’d like to leave it for the moment: If you could have had the work done on your basement for less money and get higher quality, and perhaps get it done even quicker – would you be willing to consider doing that? I know that there would be no way to know it was cheaper if you had no estimates – but conceptually: If you could get all the benefits I describe would you consider learning to do business that way? I hope we can agree that would be better all other things being equal.
We have contracts, and estimates, and predictions, and all the trappings of doing business this way because most people haven’t found a way to do things better – we fail each other, we don’t deliver as expected, we don’t trust each other, so we find ways to do business that gives us a chance to do business. So… we can try to get good at estimates, and it if serves you and your customers, more power to you.
Lastly:
Cory: “That said, while I disagree with your answer to “What can we do?” as “Nothing.”, I enjoyed reading your blog. It’s fun reading and certainly food for thought.”
My Response: I didn’t ask “What can we do?”. I asked “What can we do instead of doing estimates?”, and the answer was simply put as “Nothing”. My point being that we do not need to do anything “instead” – estimates are often simply UNNEEDED. And then I posed the more meaningful question: “How can we develop software without doing estimates?”. The nuances of this are important, and perhaps I was being too cryptic. At the time I wrote this post it seemed few were willing to explore these things. I hope I was wrong, and now I am certainly finding more people willing to entertain and even experiment with finding better ways. And some doing purely “no estimate” projects.
Overall, I have found the Agile way works well. When you have customers who see the benefits and find it works better for them than the old phased, predictive approach – then it is truly sweet.
Thanks for your kind words. I try to be fun, and I’m “all about” food for thought!!!
Cheers!
I’ve done 12+ sessions at Agile and Developer “user groups”, Agile or Dev conferences, college classes, and code camps where the main topic (or a big component) of the session was Estimates in Software Development.
Here are the notes from one such session I facilitated last year at an Open Space event (Agile Open California, North)
The title was my own. I’ve been talking about, and holding sessions on this subject for several years, and the previous time I was at this event a couple years earlier I did a session on “5 Whys on Why we Estimate”. It was a very contentious session, with almost everyone (except me) feeling estimates are important and needed for software development, and the problem is just that we’re no good at it… so we just need to get better at doing estimates. I don’t agree (as you probably know).
The “we just need to get better at it” argument sounds a lot like what I used to hear in “post-mortem” and “lessons learned” sessions in waterfall/phased project environments. A typical “take-away” from those sessions were things like this: “We missed our deadlines and had a lot of rework because the requirents kept changing, even though we have a change control process in place. We need to get better at controlling changes to requirements”. I hope you see that is very likely a bad conclusion. Essentially, that line of thinking is that “since control isn’t working, we need more control.”
I’ve been suggesting that if we keep failing at getting good at estimates even though we are working hard at becoming better at doing so (and have been doing so for countless years), then maybe we are expecting estimates to do something for us that just can’t be done. Maybe there are better ways. [Just to be clear, for me – I feel I have found a much better way and it is Agile Software Developent as presented in the Agile Manifesto (Values and Principles) – the Agile MVP.]
My position: We need to get to the bottom of what it is we want from estimates and predictions, and find a better way to get to what we really want. [HINT: What we reallly want is Sucessful Software Development Efforts (or something like that, perhaps?) . Or, as long as we are wishing, let’s just wish to win the Mega Lottery and then spend our day’s doing something we really enjoy. Like taking hikes or arguing – whatever you like.]
There were initially 17 participants. Developers, ScrumMasters, Agile Coaches, Product Managers, etc. Almost everyone had some experience with Agile/Scrum/XP/Lean. A few people joined in as the session proceeded, and a few others left.
Explore and discuss what we want out of estimates and the problems we’re having with estimates.
[NOTE: Everything will be tainted because Woody Zuill is facilitating this session, and he leans slightly toward the “No Estimates” approach.]
My approach for this session: Quickly explain my basic thoughts about the failure of estimates and the purpose of the session, do an information gathering exercise. Ask why we feel we need estimates, have the participants write their thoughts on sticky notes, group them, and then discuss.
After our gathering and grouping exercise, we began discussing what we felt this showed us. During the discussion we just “followed the flow” as ideas were expressed.
In the discussion, we were in alignment that we “need” estimates so we can plan our work, and so we can make decisions about which projects or features to work on.
Also, we still frequently ask (or our managers ask): “Are we there yet?”.
Why do we have these problems? We’ve been doing estimates for years – shouldn’t we be great at it by now? Other similar or related things that we’ve seen:
Rarely are the estimates accurate enough to really count on for release dates, costs, etc.
One reason estimates provide little chance for usefulness:
What works for me might not make sense to you. Maybe someday it will. When that happens, you will have crossed over into the land of no estimates. Beware.