No Estimate Programming Series – Intro Post
One little blog post cannot even begin to explore this complex subject. Here is a start at beginning the first part. Or perhaps, a pre-beginning.
If you really want to figure this out quickly don’t wait for me to figure out how to write about this. Just come and visit me sometime and I can quickly show you the whole thing. It really isn’t hard. Or, even easier, read the Agile Manifesto Values and Principles. It’s all there.
A Few Paragraphs You Don’t Need To Read
It’s obviously not my job to tell you (teach you, show you, guide you, coach you, whatever) how to be effective at software development. My job is to write software, and coach and work on a team that is writing software – and to deploy that working software so people can use it.
I don’t write books, sell certifications, do trainings, or provide workshops for a living. I have nothing to sell you. [I’ve occasionally done introductory sessions about Agile stuff for a company or two. I love doing that, but it is not my living or my job. ]
The company we work for makes irrigation products. Great company, great team, great product. I truly love being able to come in to work every day to be with my team and do the work we do. Point is: I make software for a living.
Estimates are NOT a deliverable
Regardless of how we define “estimate”*, it is not a deliverable in the world of software development. I’ll write another post about that if you don’t agree . (That’ll show you!)
The Basic “No Estimates” Approach
Here are the basics – in very simplified form told as a mini “case study”, based on a real situation from a long while back (not my current job) – but I’ve changed the details so no one will get mad.
Players:
- Boss – you can think of Boss as the “Product Owner”, “Customer”, or “User” or some combination of these. In any case, Boss is a “person(s)” who knows enough about the purpose of the software (or some part) to make reasonable decisions, and who is responsible and has authority for making those decisions.
- Woody – you can think of Woody as the person who guides the software development process – could be a devleoper on the team, or some other person who understands software development better than I do. We’ll call him Woody.
- Team – everyone else working on this “project”.
History:
- This was a while back, but Woody was already capable at “No Estimates Software Development”.
- This project was a new “internal” application.
- Project accessed existing databases, but also would likely need it’s own database.
- Boss had never worked with No-Estimate Software Development.
- There was a lengthy “requirements document”. It was believed that everything in the requirements doc was necessary. [Note: It was a mess, and if I had any influence I would have abandoned it].
- Team had all necessary skills and knowledge to do this work.
Approach for day one:
- Start with the huge list of way too much stuff that Boss says needs to get done (the original requirements doc, so to speak).
-
Ask Boss to select one “small” thing that she felt was very important.
-
That “small” thing becomes the “thing we are working on right now”: We’ll call it a “Story”, but on this job we didn’t use words like story. We called them features.
-
Team starts work on the Story.
Approach for day two and each day after that:
-
Team works on the thing we are working on right now. Nothing else.
-
Team constantly assesses the work we are doing, and splits the current Story if it is not easy to understand or it is not clear how to start on it. We check with Boss, but have no issue breaking things down.
-
Team asks questions as soon as they come up, and get the answers right away. Without this, projects drag on through endless muck.
-
Team completes the Story we are working on right now (or the part of the Story that we split off)
-
Team deploys the Story if it is deployable, and if it is not deployable then Team works with Boss to understand what needs to be added to it to make it deployable, does that, and then deploys.
-
Team asks Boss to pick the next thing she thinks is very important. It can be the other part of the previous story, or any other thing she thinks is important.
Result:
-
First deployment happened in a little over a week.
-
Project was deemed “done” in about a month and a week.
-
As we started delivering usable software, the users and Boss started asking for different things than had been described in the requirements document. The users and Boss started fine tuning what we had delievered – noting things that they wanted to be done differently. This is a valuable and wonderful thing. This is how requirements emerge.
-
After about one quarter of the “features” described in the requirements list/document (including the additions the users and Boss had asked for), Boss decided the project was “done enough”. The users were doing most of what they needed, and the rest wasn’t important enough to do, at least for the time being. There were other more important things to get done on another project. The users were happy. Boss was happy. Team was happy. Woody was happy.
-
Team was iterating (but not time-boxed), and incrementally delivering.
Observations:
-
Boss wanted estimates, and also wanted the work done as quickly as possible. I made the case that we could try to get some critical feature into production ASAP, and then revisit the idea of estimates later. After the first deployment, no requests were made for estimates.
-
Estimates are part of the “comprehensive documentation” that we value less than “working software”. If you don’t see that – please let me know.
-
I am sure you saw the big win, right? We discovered the needed features. Most of what was thought to be needed was not actually needed. Doing the work of writing the code in small pieces and getting it into real use made it possible to steer the work. This is the “requirements emerge” concept. I urge you and encourage you to think about it and experiment with it. It is GOLDEN.
-
Even if estimates were not liars, we just simply did not need them.
Well, that might be a little help. It’s just one very basic example. But it’s real.
This all just seems like “plain old Agile” to me. Many people tell me I’m wrong. What do I know? I’d rather be wrong about what to call this and be getting a lot of work done than the other way around. Let me live in my delusions.
NOTES:
* For the purpose of this article, the sort of estimates I am discussing are the estimates typically asked for on many software development projects where a project, a feature, or a function, or a bug fix (or where a list of features or functions) are described and people are asked to come up with an approximate cost in time, money, or effort to do the work that will be required to provide the feature(s)/function(s)/capability(ies)/bug fix(es) being requested.
Disclaimer: There are many situations where estimates can be meaningful and useful. This article is about situations where I don’t think they are typically meaningful or useful, and only in the realm of software development.
Tom Howlett:
Nice Post. What struck me most is that by delivering small storys/features regularly the “boss” stops wanting estimates because the rate of progress is so visible (and he’s probably too busy working out what he needs next). When you finish is up to the boss so there is no need to ask the team for a prediction. As you say it’s basic Agile but what you’ve described here clearly demonstrates the principles working.
Thanks
23 December 2012, 1:00 amPaul Banham:
Excellent post. I’ve worked on several projects in the dim and distant past that followed pretty much the process that you describe above. They all worked and just felt ‘right’ for everyone involved! Of course this was before the days of the Agile movement so couldn’t be classed as ‘Agile’ at the time, but looking at the Agile Manifesto it seems to fit the bill.
8 March 2013, 2:06 pmWoody Z.:
Hey Paul –
Thanks for the comments and for sharing your experience with “no estimates”. It’s my experience as well to have done “Agile-like” stuff before the Agile Manifesto and I’m sure there are many like us. I’m very glad to have the Manifesto get so much attention – they did a great job of gathering their own experiences and ideas, and those of guys like you had pioneered and providing us with a simple and clear set of values and principles we can use to judge the practices we need to get work done in our own work-places.
Cheers,
9 March 2013, 1:45 pmWoody
Chris Bird:
Interesting that he focus is on delivering software and not in delivering a solution. That doesn’t invalidate the no estimates thinking, but when we are working on solutions requiring a lot of users who need to be trained, we do need to get the new system training scheduled into already busy people’s work. Help desk solutions, reservations agents, ticket agents,… all large constituencies of people that need consistent practices in the use of the systems/solutions being produced. So when trying to figure the total cost of deployment of a new solution, we have estimating work to do. That kind of estimating deals with both cost and time. It is broader than estimating software.
15 April 2013, 9:35 amWoody Z.:
Hello Chris – thanks for your comments!
On your training example, how much ahead of time do you need to schedule the training? One week? One month? One year? If training is one of the things being delivered, it should be developed along with the project, and ready for use as the product is rolled out for use. I see little reason estimates are needed for that.
However, it points out a serious issue for any organization that thinks in terms of “big bang delivery”. The Agile principle of “Early and Often Delivery” is clear: You must find a way to deliver your software into production and “real use” as soon as possible with some small part of the desired functionality. If you can’t do that, why bother following an Agile approach at all? Simply resign yourself to a long, likely-to-fail project and do the best you can. On the other hand, any orgranization that focuses on frequent delivery of useful software that is easy to enhance, easy to change, and easy to maintain will soon find they can solve the sort of issues you described.
27 April 2013, 9:33 pmAaron Gray:
Great stuff.
Been trying to advocate this sort of development style with my customers and teammates lately. I was so refreshed to learn about this No Estimates movement and other people pushing the same ideas. Keep up the good work.
26 February 2015, 4:11 pmAntek Baranski:
I was personally lucky to run a development team like this once and the resulting software project delivered exactly what the end users wanted, on a shorter timeline and under budget.
I believe that today this ‘management’ approach is referred to as Kanban.
The caveat that I put on this approach leading to success for a team is the team’s discipline, if the team is not able to focus on “what the most important thing” that the ‘pointy hairs’ want (for whatever reason) then this software development process will end in disaster for all parties.
27 February 2015, 12:23 pmKamal:
Hi,
This sounds like a good approach when developing a new application, or perhaps updating an existing one, but this probably only works well when the application is standalone.
I’d love to see a good way to approach agile development in a complex enterprise environment, where there are multiple applications, teams, etc. These systems are often tightly coupled, with defined interfaces that have a low tolerance for breakage and self-healing/discovery.
How do you handle budgeting? Typically, we have many projects (tied to marketing products, regulatory driven changes, platform or performance upgrades, etc) that are targeting multiple platforms for each project, simultaneously.
Enterprise development is a really important challenge in the world of software development, and its not going away anytime soon.
As a professional PM, I am often frustrated by the limitations of the waterfall process, but I don’t see a real alternative in the agile world either as applied to the challenges above. We have experimented with hybrid models, where new “back-end” features go through a waterfall process, and then front end features (ie mobile/web) leverage those backend features through an agile process, but this still means that we typically need 9 months from concept to implementation for these back-end features.
MKS
27 February 2015, 1:10 pmJosh Seiden:
I’m a big believer in this tactic, and we implement it in our product development consulting work. Here’s a blog post that explains our process: http://neo.com/2015/02/05/how-to-scope-design-for-agile-and-lean-projects
27 February 2015, 2:09 pmBen McEvoy:
There are a number of elements to your project which allowed this approach to work, without which, I imagine you may have more trouble:
1) You had already “won” this work. There are situations where a customer requires a price up front, as part of their decision of who to award the work to. If you refuse to give a price estimate, you don’t get the project.
2) You could negotiate scope. There are situations where definition in scope forms part of the legal contract for delivering the solution. Particularly in the public sector, where public money is being spent, the supplier will be legally held to account to deliver all the agreed scope. This relates to #1 – the cost-scope relationship is defined in a contract which you’re legally bound to deliver.
3) Your project involved relatively well-known technologies. This is a major source of “unknown unknowns”. You didn’t need to plan your architecture or framework in advance. If you had been working with relatively unknown technologies, you may have still been trying to get widget x to talk to widget y in week 10.
4) Your project was relatively small. Software project complexity is exponential as it scales – a six-week project is orders of magnitude simpler than a six-month project for the same size team. Estimating as an activity is linked to planning/architecting – you’re solving problems, at least conceptually, as you do it. A “just start doing it” approach may work well for relatively small projects, because the complexity is small enough for your team as a whole to understand and deliver. There’s a bit of rework, but not enough to offset the benefits of immediate productivity. Once the project gets larger, the benefits of spending some time planning in advance offsets the huge amounts of rework or overlapping, blocking dependencies you encounter by just “doing whatever the boss says is most important first”.
5) This was an internal project. This means you could easily “asks questions as soon as they come up, and get the answers right away”. Your feedback loop is very short. In other projects, the feedback loop is very long – in other cases, practically non-existent. In these scenarios, you often can’t just do one thing, or just do the thing that’s most important – you’re roadblocked or held up waiting for feedback from your users and need to be able to work on multiple items in parallel.
6) You had a single, accountable stakeholder, who defined priorities, and when you were “done”. In the world of product development, where you build something you hope will have a very broad appeal, with lots of different users with competing needs, you never have this luxury. You won’t be able to easily prioritise one feature over another, because your users will almost certainly not share a common, single vision of what is most important – and they’ll never agree on when you are “done”.
Don’t get me wrong – I love the idea of emerging requirements and it would be great to always be able to take an approach which is flexible enough to allow developers to just get stuck in and work it out as they go along. I just don’t think this idea scales, nor is it the best approach for every project.
1 March 2015, 5:41 pmWoody Z.:
Hello Ben,
There are always a lot of reasons that something “won’t work in my situation”. What I’ve shared here is a simplified example of the sort of things that we can have happen if we pay attention to making things better. This is a distilled example. It seems to me that you are looking for all the reasons this can’t work outside of the example. It’s easy to say “this can’t work in my situation – your situation is somehow ‘special'”. I’d like to encourage you to think about ways you could overcome each of your objections.
Good luck to you!
Cheers,
9 June 2015, 2:20 pmWoody
Ankush:
Ben,
You’re right on all counts. This example wouldn’t work if we apply the constraints you describe. But this wouldn’t be a failure of No Estimates, but the failure of the parties involved to understand and embrace a better approach. Agile requires ALL the players to change their mindsets first, or the output is zero. In my organization, daily meetings happen just the same old, boring, unproductive way in the name of Scrum. Can’t help those who don’t want to change!
As for architecting something in advance, you can use the exact same approach. Come back to the business guys with finished architecture of a small module, and then add another. If your architectire was sound, all the units would fit together nicely in the end. Yes, some time would be spend in the very initial thought, but even then we can focus on taking decisions one at a time.
Hope it helped.
28 February 2016, 4:33 amAnkush
Pavel Najman:
Hello,
I was reading your posts about #NoEstimates for a while and I wanted to share my point of view (this is probably not the latest post but I hope it does not matter).
First of all, you made my think, that is great. I started to evaluate what real impact estimates have on our project and conclusion was easy (the feeling was there for some time)- NONE. I think that if a monkey would randomly assign values from 2-21 to our user stories, we would be where we are now anyway. The management is not following them that much either. They are just happy when some functionalities are done. Well, they are asking for them time to time but I did not see that they use them in any reasonable sense.
However(! 🙂 ), we maybe do not need estimates, but I think we need estimating. During this process we very often find that the scope of a user story is understood differently by different team members. When someone votes 2 and someone 21 on the same user story, it is obvious that we need a discussion. We are not co-located team and communication does not come that easy. Simply, we use estimating as a check for understanding of user story. Before we did it we were in much more bigger troubles then we are now (at least that is my honest believe).
So my question is, when working on project without estimation, how do you check for common understanding of the user stories? Separation into tasks?
7 July 2016, 3:17 amWoody Z.:
Hello Pavel,
Voting on the size of a story is merely one way to attempt to spark a discussion. I haven’t seen a need to do size estimates to explore our different understandings of a story – as simply looking at the information available about that story and talking about it has worked very well for that. For example, inspecting the specific attributes of a story, such as the possible modules it is dependent on, generally leads to a useful discussion and the building of a shared understanding.
Regarding “separation of tasks”: If it is important to us to separate out the tasks we can simply start separating out the tasks – they are there regardless of what we think the size is. We’ll quickly learn if the story is at a workable size without every having to make a quick guess about that size.
Cheers,
1 February 2017, 10:27 pmWoody
kostas chairopoulos:
Hi Ben,
I think there is an important point as missing. Software development is also on projects, many of them through an exhaustive tender process. So if you want to take the job you need at least a rough estimation for your cost, resource etc. Estimation is not about precision (you cannot be 100% accurate but also you cannot be 70% wrong). To use an analogy, estimation is like a travel plan trip (destination, cities to visit, days of stay, you booked the flight tickets but not the hotels to stay)
Thanks
29 November 2016, 4:50 pmKostas
Woody Z.:
Hello Kostas,
I often hear this, and there are many “projects” contracted this way. Estimates do not server this model either, as the model itself is flawed. Merely because “this is the way business is done” doesn’t mean it is the way business must be done. The belief that this common current model is working well hides or ignores the problems.
Cheers,
1 February 2017, 10:19 pmWoody