Life, Liberty, and the Pursuit of Agility

There are a number of definitions people use, and I’ll give you mine here. Perhaps this is less a definition and more how I think about legacy code.

Legacy code is code that is doing some real work.  It exists because it is bringing value.  We work on it because we want the code to keep bringing value.

That is, any code that is in use is legacy code. We inherited it from yesterday.  Someone else might have written it, or we might have inherited from our self.  Regardless, it is meaningful to us because someone needs to use the functionality provided by the code.

I often hear people disparaging Legacy code.  When the code is bad, it can be hard to keep from complaining. I don’t. I honor the code, and the developer who wrote the code.  It is bringing value, and that in itself is a tremendous achievement.  Regardless of how good or bad it is, they (or I) got it to work well enough to be of use.

The developers(s) who created the code worked under imperfect conditions.

There is a Google Group on #NoEstimates. I joined it, and have visited it every now and then, but I have not participated.

I do not participate much in this sort of discussion group

Here is a bit of a quote (from a wise and well respected person) that goes a long way towards explaining how I feel about the #NoEstimates (and most) discussion groups. I have paraphrased it almost out of recognition to better clarify my position.

“Replacing a face to face talk with a Google Group is like replacing a hug from your Mom with a face full of pepper spray”.

Conversations

It is rare that any two people agree completely about anything, let alone something as complex as the endeavor of managing software development. I don’t expect anyone to agree with anything I have to say, and it is a rare yet wonderful experience for me when I find someone who sees things in a way that is more or less in alignment with the way I see things. I don’t expect it.  However, I also enjoy having conversations with people who see things differently.

While I enjoy conversation, there is very little actual conversation going on in the typical discussion forum or Google Group.  Unless a group is actively moderated, it quickly degrades into something that is much more anti-conversational than it is conversational, more harmful than it is useful to gaining a shared understanding.  Perhaps they are not meant for conversations and understanding.

Recent forays into the land of Discussion Boards

I’ve recently read through several topic threads on various discussion boards at LinkedIn or Google Groups and it is typically a painful and disturbing experience for me. Overall, these threads are not productive conversations as far as I can tell, and seem to simply be a jumbled mess that is nearly impossible (for me, anyway) to decipher.  Some people must get value out of this sort of communication forum, but in general I don’t find them conducive to meaningful conversation.

Often these forums seem more like an opportunity for people to say “You don’t understand”, and then provide some rebuttals and clarifications that provide fuel for the next “You don’t understand” response. (I am using “you don’t understand” as a placeholder for all similar [and often less kind] responses).

Even worse, each response contains snippets and quotes (sometimes HUGE snippets or the entire content) of what someone in a previous post stated, and then intersperses the snippets/quotes with rebuttals and clarifications and “You don’t understand”, and “that is not how it is in REAL life” statements.  And even worse than that, sometimes the snippets and quotes accumulate from response to response building into an incomprehensible cacophony of confusion where there is no way to unravel who is trying to say what about which.

All in all, while this has been going on for a very long time (years and years), and while I participate in this sort of forum every now and then, I see little value in it for me.  I have enough to do already.

NOTE: The mini-forum of Twitter, while in my opinion is also not conducive to conversation, at least limits the amount of cacophony. Rather than the “continuous ringing of huge, deafening alarm bells” it’s more like the “continuous jingling of tiny, less deafening alarm bells”, and there just aren’t enough characters to gather garbage snippets.  Anyhow, I’m pretty much on board with the #NoCacophony movement.

I prefer face to face, one-on-one conversations

I prefer to have engaging and meaningful conversations with people who are willing to talk with me.  The most productive, useful, and pleasing of these experiences for me are in one-on-one, face to face sharing where the both of us are sitting at the same place, or taking a walk together.  Sometimes having these conversations in very small groups with everyone in the same place is just as good.  I have found this to be very fruitful to me for my own learning and exploring, and for vetting and critiquing my ideas and thoughts. I can only assume that this works well for at least some folks since I frequently meet with others this way.

While this “in the same place” mode is great, it is not always possible.  So, my favorite alternative (at this time)  is to have a conversation via video Skype or Google Hangout (or whatever technology that works).

Sometimes, the only viable approach is to make a voice only call, but this is in a very distant 3rd place.

Besides those 3 basic approaches (same space, via video chat, and voice only call), there are a few other variations that can make things better (like having a screen share in a remote conversation).  But that is beyond my purpose in this little post.

Why do you suppose face to face, one-on-one conversations work so well?

Why does the one on one, face to face (or small group) style work so well? I’m sure someone probably knows, and I have my ideas, but that is not important to me. What is important to me is that it works well for me, and it is my preferred approach to learning and exploring, vetting and clarifying.

When we make it our goal to find a way to understand each other it often requires that we change the way we are saying something, or the way we think about something.  That is, we might “change our mind” based on the new information and understanding we are gaining.  Our eventual understanding will likely be different from where we started for both of us.

When our words and thoughts are “locked down” in a forum thread, making changes to more clearly communicate our intention is (typically) not possible – we have limited “edit” rights.  Even worse, the path to shared understanding seems to be thwarted, and old misunderstandings just get brought up again and again, sometimes by people who were not part of the original conversation and don’t understand what has been covered, or the path that led to the current understanding.

With spoken communication we tend to just move on.  We say things like “I see what you are saying”, “You are right! How did I miss that?”, “Now I get it”, “Oh… you meant this instead of that”, “Of course!  That makes sense to me now”, “Here is something I think I wasn’t considering…”,  and so on.  I don’t see these phrases often in the discussion boards.  This isn’t always true, but I’ve found that spoken conversations tend to be more conducive to making progress towards a shared understanding.

Regardless… you must find the communication style that works for you.  If forums and groups and discussion boards work for you, then GREAT!  You just won’t see me join in much.

On the other hand, let me know if you want to chat sometime.

Cheers!

Woody

I’ve seen a lot of orgs/teams/groups/whatever (I’ll call them orgs for now) doing things that aren’t working, and yet proceeding as if they are working.

I’ve seen a lot of these orgs move on to “better” starting from wherever they are by learning to reflect, tune, adapt, adjust (and some other things).

I’ve seen number orgs revert or “backslide” as well – both slowly over time, as well as almost instantly in some cases.

Whatever the situation, those of us who want things to be “better” need to take action, and that will never stop.  At least I’m pretty certain that will never stop.

I take it as my responsibility to make things better in my profession and in the places I work.  I act on this responsibility by exploring possible “better ways”, implementing “better ways”, and influencing things to the degree I can.  To that purpose, I constantly work on improving my ability to influence things for the better – sometimes I have success, sometimes I don’t.

To me, this is “Agile”:

An approach that works well for me is to always take rapid, never-ending baby steps to “better”. I constantly reflect, tune, and adjust. When things work, great. When they don’t, it’s easy to adjust or take back that baby step and try some other step.  Sometimes I take leaps, but I prefer taking baby steps.

For me, there is often a “Lofty Goal” that represents a vision of a better future.  It will change over time, and any specific “lofty goal” can grow, morph, or fade away over time depending on what I discover as I take more baby steps.

I want to always have the option to make mistakes, and to pay as little as possible for those mistakes.  Sometimes I must pay a lot for our mistakes. Still, I prefer to find ways to keep the price tiny when I can.

Being resolute to work with others to always steer to the next possible “better” seems to work nicely.  I fail in many ways, but it is not the ultimate failure of thinking and acting as if “I can’t make a difference”.

The Values and Principles of the Agile Manifesto are the foundation of my idea of what “Agile” is, and what it is to me.  I have expanded on these ideas for my own work, and I don’t see them as being “static”, but rather a somewhat firm yet dynamic set of guidelines that are easy to apply for evaluating my thinking about the business of software development, and for my exploration of software development practices and techniques.

I know what works for me: I take the Agile Manifesto seriously, and I constantly Reflect, Tune, and Adjust.  I think of this as “Pure Agile”.

Cheers!

I have made plans to speak at several upcoming conferences and events.  If you are attending any of these, I’d love to meet up with you and talk about Agile, Mob Programming, #NoEstimates, #BeyondProjects, or just about anything please look me up!

• Agile Open Southern California 2014 – UC Irvine, Irvine, California – Sept 11 and 12
• Java One 2014 – San Francisco, California – September 28 through October 2
• Stockholm Software Craftsmanship Community – October 14th
• Mystes – State of the Art Practices in Agile Software Development  – Helsinki, Finland – October 23 and 24
• Tampere Goes Agile 2014 – Tampere, Finland – October 25
• Oredev 2014 – Malmo, Sweden – November 3 through 7

• Santa Barbara – January
• Mitchell DevCon 2014, San Diego, CA
• Twitter – February
• Tagged – February
• Mach One @Missoula – Missoula, Montana – April 11 and 12, 2014 – Lightning Keynote, Mob Programming, Beyond Estimates, Beyond Projects
• Winnipeg Agile Users Group – May – Continuous Discovery, The Value of Pure Agile
• Winnipeg Mob Programming Workshop
• Agile Games 2014 – Boston, Massachusetts – June 2, 3 and 4, 2014 – Mob Programming Workshop
• Agile New England – Mob Programming and Leadership – Enabling the Self-Organizing Team
• Scrum Day San Diego – San Diego, California – June 13, 2014 –
• Agile Roots – South Jordan, Utah – June 19 and 20 (and bonus day on 21) 
• Agile 2014 – Orlando, Florida – July 28 through August 1, 2014
• San Diego PMI – Mob Programming -I’ll get the details when I can.

Advanced Practices for Leading Agile Retrospectives: New Techniques and Activities

with Diana Larsen

When: Thursday, March 6, 2014 from 9:00 AM to 4:00 PM (PST)

Where: Marina Village, San Diego, CA

Visit the Eventbrite page for this event at http://dianalarsenworkshop.eventbrite.com

Spend an entire day with Diana Larsen exploring, learning, and practicing Advanced Practices for Leading Agile Retrospectives.  Agile Retrospectives improve any project or process — building on a team’s immediate past experience of success and failure. Smart teams and organizations hold Retrospective meetings iteratively, throughout the work cycle and at important release milestones.

In this workshop, Diana Larsen will introduce advanced uses of the Flexible Framework for Retrospectives and show how to incorporate the Five Rules for Team Learning in your Retrospective designs. Participants will share new team activities to enlarge your repertoire and gain practice in designing and facilitating effective Agile Retrospectives.

The Day:

1. Introductions – including practice with new activities to “Set the Stage”
2. Agile Retrospective Purpose & Group Learning, Thinking, and Collaborating on Improvement Actions
3. Applying Five Rules for Learning in Retrospectives
4. Explore and practice new activities to “”Gather Data”, Generate Insights”, “Decide What to Do”, and “Close the Retrospective”
5. Retrospectives as Adaptive Action for “Responding to Change”
6. Designing Agile Retrospectives for Continuous Learning and Improvement
7. Next Steps

Our Facilitator, Diana Larson: 

Diana is a founding partner of FutureWorks Consulting. She is considered an international authority in the areas of Agile software development, team leadership, and Agile transitions.  She co-authored “Agile Retrospectives: Making Good Teams Great”; “Liftoff: Launching Agile Teams and Projects”; “Quickstart Guide to Five Rules for Accelerated Learning”; and most recently the Path Through Agile Fluency model, described in the article, “Your Path Through Agile Fluency: A Brief Guide to Success with Agile” at www.agilefluency.com

Tickets Available on Eventbrite: 

We are keeping the price for this event as low as we possibly can.  You can attend for only $99 for a “Bring your own lunch” ticket, or $114 and we’ll provide a Box Lunch.  Also, we have set up a “5 for the price of 4” discount if you have a group from your company who would like to attend. 

Visit the Eventbrite page for this event at http://dianalarsenworkshop.eventbrite.com

This is  a little story about how I learned the right way to manage people

My First Job: Age 13 or 14

When I was a kid, if I wanted something (like a slot-car, or a pellet gun), I could wait for my birthday or Christmas and hope my folks could afford to buy it for me, but beyond that it was pretty much up to me.  So I pulled weeds, sold garden seeds door-to-door, sold Christmas and greeting cards door-to-door, sold grapes on the roadside, delivered newspapers as a back up for friends who had paper-routes.  Typical kid stuff for that day.

But those little jobs never brought in much money for the amount of work, so I decided to get a “real” job.

There was a plant nursery about a mile from my home, and they occasionally hired teens to water plants, move stuff, mix soil, plant seedlings into nursery cans, and similar nursery work.  So, to make a long story a bit shorter: I asked for a job, and the owner of the nursery, Mr. Smith (his real name), hired me.  He knew my folks.

Lesson One: Treat everyone nicely

The day I started, the Mr. Smith introduced me to Bill (also his real name).

Mr. Smith: “It’s great to have you join us here.  Bill is going to show you how to water the plants today – it’s not difficult, but there are a few important things you’ll need to know. You won’t have any trouble with it.”

Lesson Two: Continuous Improvement

Mr. Smith: “However, another part of the job is for you to think about what you are doing, and look for better ways to do things.  Keep track of your hours, and next Saturday when I pay you for the week I want you to tell me one way to do things better.  Look for problems, and think about how we could deal with them.  Okay?”

Me: “I’ll do that”

Mr. Smith: “Great! Bill will take you out and show you what you’ll be doing”.

Lesson Three: Give people a sense of the importance of what they do.

Bill: “Hey Woody, I’m glad Mr. Smith hired you. We’ve been too busy to take care of everything and we’ve got a very important job for you. Let me show you.”

We walked out into the nursery, and Bill grabbed a hose with a watering wand attached to the end and handed it to me.

Bill: “You are going to water the plants. It might seem simple, and in some ways it is.  But these plants depend on us for their lives – they can’t get water without us.  We water most of these plants every two days.  Each plant is in a can just large enough to provide the room it needs for it’s roots.  When you water a plant, give it just enough water to wet the roots completely, but no more – we don’t want to waste water.  However, if you give it any less, the plant may possibly dry out too much by the next time we water, and that stresses and weakens the plant.  Got it?”

Lesson Four: Most People Are Smart Enough To Figure Things Out.  Let Them Do That.

Me: “Sounds good.  How do I know what’s enough water?”

Bill: “Well, it’s a bit tricky.  You have to check some of the plants after you’ve watered a row.  Go back and see that a little water is dripping out the holes in the bottom.  That shows you the whole root-ball has gotten some water.  Keep your eyes open.  If some of the plants start wilting a bit – you know you have to adjust your technique.  You’ll figure it out”.

Me: “How do I tell if a plant is wilting?”

Bill: “Each species is a bit different, some need more water, some need less.  It’s just something you get used to over time.  You’ll figure it out.”

And so I did.  I watered the plants.  Hundreds and hundreds of plants. I dragged the hoses around, and I carefully watered each plant.  I also went back to check some of the plants when I got to the end of a row to make sure I was doing things right. I watched for wilting plants.  I was careful to not waste water.  IT WAS GREAT.  I was outdoors all day, I was doing an important job and it gave Mr. Smith and Bill more time to work with customers, pot up plants, make deliveries and so on.

Lesson Five: Make Everyone Welcome, Not Just Feel Welcome, But Truly Welcome

After about 2 hours on the first day, Bill came over to me and said it was time for a break.  A break? This was great!  I get to work, I’m making money, AND I get to take a break.  This was heaven. [Remember, I was just a kid and had no idea what work was actually like – I was just excited to be working]

Bill: “Hey Woody, let’s take a break.  We take about 15 minutes every few hours to get in the shade and have a soda.  Let me show you the soda machine.”

Soda machine?  I had no money with me.  I had no money, period.

We walked over around the back of the nursery to one of those old fashioned refrigerated coin-operated soda bottle vending machines with bottles of soda you pulled from a slot after opening the big, heavy, metal lid.  They had rigged it so you didn’t need coins.  Just pick out the drink you want.  SWEET! And they had Vernors Ginger Ale.  I was living high on the hog!

We sat and talked about plants, and the weather, and how wet you get watering plants. Bill was sincerely welcoming me.  He treated me kindly that day, and every day I worked with him. We were all there to work together, each of us depended on each other, and it was expected we would be polite and respectful in all things. I felt like I was part of the family already.  And that never stopped.

Lesson Six: Trust People

At the end of the week, I went to Mr. Smith to get paid.  I handed him my notebook where I had written my hours for each day.

Mr. Smith: “Well, how was your first week?”

Me: “I loved it”

Mr. Smith: “Thanks for keeping track of your hours. Please add them up, and multiply by $1.25 per hour, and then go get your money out of the cash box under the front counter.  Write down the total on a slip of paper and put it in the bottom of the cash box  – I need it to do the taxes.”

I am not kidding.  Maybe it was a different time back then, but I have a feeling it was much more than that.

Lesson Seven: Give Meaningful Feedback

Well, I got my money and went back to Mr. Smith.

Mr. Smith: “So, how did it go this week?  You getting a feel for the work?”

Me: “It went pretty great.  I really like working with Bill.  I like being outside all day.  I like the pay.  I like everything.  I think I figured things out, and I made sure every plant got plenty of water without wasting any.”

Mr. Smith: “I’m glad to hear that.  We’re glad to have you here – it’s really helping us out. There is one thing we need to work on. I noticed you are very diligent, and you didn’t allow any of the plants to dry out, or even get stressed.  That is exactly what we need.”

Me: (I couldn’t say anything I was so proud, but I was BEAMING!)

Mr. Smith: “But… Bill and I can each water the plants in about half the time it takes you. I’d like to see if you can do it in about half the time too.”

Lesson Eight: Let The Worker Figure Out How To Do The Job Better

Me (a bit saddened, but only a bit): “I thought I was doing a good job. I don’t want to kill any of the plants.  How do I deal with that?”

Mr. Smith: “Well, the trick is to give each plant the exact amount of water it needs.  No more, no less.  We can’t measure the amount, because each plant is a bit different even for the same species: slightly different age, slightly different soil mix, a little more sun here, a little more shade there… that sort of thing”.

Me: “So how can I tell?”

Mr. Smith: “You’ll figure it out.  Next week, I expect you to be able to water the whole nursery in half the time.  It took me time to learn – it will probably take you time too.”

Me: “I’ll do my best”

Mr. Smith: “I have a feeling that your best is going to be better than my best.  Let’s see how you do. So, remember I asked you to keep your eyes open for problems?  Did you find any?””.

Lesson Nine: Continuous Improvement Continued:

Me: “I remembered to look for problems, and I noticed that there was a leak in several of the hoses. It was wasting water and making the paths muddy.”

Mr. Smith: “What should we do?”

Me: “Well, I showed it to Bill, and he showed me how to fix the leaks.  We did it yesterday”.

Mr. Smith: “That’s great.  Let’s go get a soda.”

Lesson Ten: Reflect, tune, adjust

The next day, I tried to figure out how I could work faster and still water all the plants. One problem was that I still didn’t know how much “just enough” water was.  I knew when the plants had too much water – it ran out of the can, and into the gravel – but without letting the plants get under watered, how would I know what too-little looked like?

My solution was to set aside one plant out of each 100 or so, a “test” for me to experiment with.  I set them next to the edge along the row, and gave those plants about half the amount of water I had been giving all the other plants – 1/2 because I was taking twice as long as Mr. Smith needed. That seemed like a good starting point. At the end of the day, I checked those I had set aside along each row (about 200 plants or so overall) to look for wilting, dry soil, stuff like that.  I lucked out.  Each was fine.

The next day, I checked the “test” plants, and most of them were doing just fine… so, I had my solution: I had been overly concerned about too little water, and it was taking too much time.  Now I had “dialed in” the right amount.

So… the rest of that week, I used the new “correct” amount of water.  I made sure to stay alert and keep my eyes open for wilting and drying – and it went well.  Every now and then I’d spot one that was starting to stress, and I’d give those plants a bit more water with a watering can I carried for the purpose.  Overall, it went very well.

Lesson Eleven: Continuous Improvement Revisited, And Listen To What The Worker Says, And… More Lessons

At the end of the week, I went in to Mr. Smith to get my pay, and report on improvements I wanted to suggest.

Mr. Smith: “Hey Woody.  I was watching you this week.  You really did a great job – all the plants look healthy with just a few dry ones here and there, and you got your time right where we need it to be.  Thanks!”

Me: “Thanks for noticing, Mr. Smith.  I was really glad I was able to figure it out.  I was worried I wouldn’t be able to”.

Mr. Smith: “Well, I had no doubt.  You care about the job you do.  You pay attention, and learn from what you see.  So.  What did you figure out we can improve on this week?”

Me: “I’ve been dragging that hose all over the place.  There are only two faucets, one in the front and one in the back.  I have to drag the hose back and forth, and up and down each row.  I think it is taking a lot of time and effort just to do that, and it is wearing out the hoses.”

Mr. Smith: “How will you fix that?”

Me: “You want me to fix it? We’ll need to dig some ditches and lay some pipes, and put a faucet at the end of each row, and get a hose for each row.  That’s a lot of work. I’ve never done that before.”

Mr. Smith: “Can you figure that out?”

Me: “I guess so. But it will cost a lot of money to do that.”

Lesson Twelve: Engage And See

Mr. Smith: “How much time do you think it will save?”

Me: “I think I’ll save a lot of time – maybe an hour or two each day”

Mr. Smith: “How can you prove that to me?”

Me (after thinking a while): “Well, I could put in one new faucet for one row, and keep track of how much time that saves me.  Then multiply that for each row.  Will that work?”

Mr. Smith: “It might. Give it a try”.

Lessons Thirteen Through Infinity

That is just the start of the things I learned about the workplace and the way things should be.

So, What Happened Here?

I could go on like this for a long time.  This was my first two weeks working at the best job I’ve ever had. Kind people, trusting people. They were trustworthy, and expected me to be trustworthy.  They worked hard and expected me to work hard.  Even more, they worked smart, and expected me to work smart.  I was 13 (or 14, I don’t quite remember).  I was just a kid.  They treated me as an equal, and expected out of me as much as they expected out of themselves.

Worst thing: This spoiled me for all other jobs.  I worked there after school and on weekends all through high school, and after high school I continued on and off for a few years.  I learned a lot – about plants, and more importantly, about work, how to treat people, how to “manage”, and how to “lead”.  And how good it could (and should) be.

I haven’t been able to live up to my memory of Mr. Lawrence Smith, and I have fallen short of what I think he expected of me and my life, and in how I treat others.  I miss him still, all these many years later (he died long ago.)  God Bless You, Mr. Smith – you were, and are to me, one of the best.

I think this is a simple question:

In your software development efforts, would you eliminate estimates* if you found they bring no value? Put another way, if it costs (time, money, effort, whatever) to do estimates, and you get no return for that expense AT ALL, would you still do those estimates?

Few Are Willing To Answer This Question Outright

I’ve asked this question to about 50 people (face-to-face, in conference sessions, in “Skype” conversations, and in other forums) and here are some of the typical things I hear in response (note that these “answers” don’t answer the question at all):

• I still like estimates, even if they bring no value, because they help me think about my work
• We NEED estimates to make decisions
• I just use them to determine velocity
• My boss still requires them, even if I think they are waste
• We need estimates to be able to bid jobs – our customers require them
• It depends on the context

I rarely hear the answer I originally thought I would hear:

• Dang right I’d eliminate them! Anything that is a cost that brings no value is WASTE and I would rather spend my money some better way. I won’t keep any process, practice, procedure, policy, or plan with no pay-off. (And quit spitting, will ya?)

I hear a more reasonable answer when I ask the question like this:

Same question (more or less) without the word “estimates”: If waste is anything that has a cost but has no value, would you eliminate waste from your process were you to find it?

Most of the time the answer I hear to this is “You bet I would”.

It seems to me, for the time being anyway, that there is some sort of problem people have when I insert “estimates” into the conversation.

The question is hypothetical

Why are few willing to actually answer the question, and instead “dodge it”?  I don’t know.  I might be asking it wrong.

I pose hypothetical questions to myself all the time.  One of my favorite “thinking tools” is to take the opposite side of some belief I have. For example: I love pair programming.  It feels right, helps me work faster and stay on track, accelerates my learning, gives me confidence in my code, things like that.  So, I will challenge myself and ask: What if I’m wrong? What if pair programming is a wasteful practice? How could I prove that? What would I do if I did  prove it?

Exploration of questions like these help me grow my ability to do continuous improvement. At least it feels that way to me.

I’m Dedicated To Making Things Better

I am always on the alert for waste. It comes in many forms, yet it is often hidden, or disguised as something of value. It’s fun, interesting, exciting even, to find and eliminate waste.  And, it is just plain good business.  A penny saved is ten cents earned.  If you don’t understand how that works, I’ll explain it some other time.

I find it helpful to remember the old Einstein saying about insanity, which I paraphrase here: We keep doing this, it doesn’t seem to work, we need to do better next time, we try to do better, things never get better. Perhaps “getting better at this” is failing, and “this” is not the right thing to do at all.

Once I’ve opened my eyes to the possibilities, I usually find big improvements are going to start happening.  The old thinking and old practices that have been blocking me from “better” are gone, and I can allow myself to imagine the possibilities, and to start innovating.

All I ask

So here is what I ask: Think of it like debate class.

The proposition: “Estimates are waste and we must eliminate estimates in software development”.

Take the affirmative position supporting the proposition, and prepare your arguments.  Suggest ways you could prove that estimates are a waste. Propose how you could replace estimates in your process.  Imagine you could convince your customer to hire your company without a bidding process – perhaps something more like choosing a doctor than buying a car.  Stuff like that.

We won’t really debate – this is just an exercise.  Are you willing to stretch your thinking a bit?  It shouldn’t take long. And you can quit any time and watch TV instead.

Some other posts I’ve written on the topic of estimating:

• Estimation is Easy and Useful: Estimate a game of Chess
• Do Estimates Do What We Want Them To Do?
• No Estimate Programming Series – Intro Post
• No Estimate Approach For End-Of-Life Legacy Support
• Notes From A Conference Session On No Estimates in Software Development
• A Comment And Response from Estimate Chess Post
• Can We Code Without Estimates?

Cheers!

NOTES:

* For the purpose of this article, the sort of estimates I am discussing are the estimates typically asked for on many software development projects where a project, a feature, or a function, or a bug fix (or where a list of features or functions) are described and people are asked to come up with an approximate cost in time, money, or effort to do the work that will be required to provide the feature(s)/function(s)/capability(ies)/bug fix(es) being requested.

Disclaimer: There are many situations where estimates can be meaningful and useful.  This article is about situations where I don’t think they are typically meaningful or useful, and only in the realm of software development.

Lately I’ve seen/heard a lot of discussion about deadlines – and it seems that they are often misunderstood.  Fortunately, I am often misunderstood, and I misunderstand most things… so it seems perfect I should write a blog post about this.

I Worked As A Flyboy At A Newspaper In My Teens – [you can skip this part]

You might have heard of newspapers.  They used to have these things where you could read news that was printed on paper – thus “newspaper”.

In my teens (in the late 1960’s) I had a night job as a “Flyboy” and printer’s assistant for a small newspaper printer who had their own web press (a web press is a huge machine where you put in 1-ton rolls of paper at one end and output printed and folded newspapers at the other end).

What’s a flyboy? It is a boy who stands at the “output” end of the press and picks up an arm-full of newspapers as they come off the press, quickly “jogs” them into a neat stack, and bundles and ties them on a bundler/tier.  “Fly” because the papers “fly off the press”, “Boy” because no reasonable adult would take this job unless they had no other choice.

A Few Things About This Job  – [you can skip this part]

• It’s LOUD: you will soon loose your hearing.
• It’s DANGEROUS: the various parts of the machines (cutter, folder, bundler/tier, etc) are all capable of removing parts of your body, mostly fingers or hands.
• It’s DIRTY: the ink is still drying as you handle the papers. You get completely covered in toxic pigments that you have to scrub off with a scrub-brush at the end of a shift, and you’re breathing paper dust and toxic fumes the whole time.
• It’s a NIGHT JOB: Or, very often it’s a night job because the papers go out in the morning.
• It’s HARD: It takes a great deal of constant physical exertion so you are guaranteed to be worn out at the end of the shift.
• In other words, at the end of the day you are tired, dirty, injured, and nearly deaf – much like computer programming.
• You don’t want me to tell you the bad things about this job.
• This has nothing to do with the point of this story.

Besides being a flyboy and a printer’s assistant, I also would do typesetting for some of the less important content, and operate smaller presses to print advertising flyers that get “stuffed” into the papers at some point, and help run the cameras and to do various layout, stripping, and other tasks.  Most of this stuff is now considered a “dying trade”. And good riddance, I suppose, but I loved it – it seems I always loved books, printing, machines, noise, lettering, typesetting, doing stuff, craft, high skills, hand work, hard work, and late night radio.  Especially the shipping reports from England that come on at 2:00 am PST.

Point is:

The point is I don’t know a lot about the whole process except what I learned when I worked in this environment for about a year, and I think I remember just enough to make my point (hopefully).

Getting the newspaper ready for printing each day requires a lot of specific things get done ON TIME

All of the following list of work must get done on a very tight schedule every day:

• Before you can print you must make the printing plates.
• Before you make the plates you must do the layout, camera work (separations, halftones, etc), and stripping
• Before you do the layout, camera work, and stripping you must do the typesetting.
• Before you can typeset you must have edited copy
• Before you can edit copy you have to decide that a story merits getting into the newspaper
• Before you can decide which stories merit getting into the newspaper someone must write those stories (or at least the headline and some copy) and take photos
• Before you can write stories and take photos the reporters and photographers have to go to the scene of the story and gather news, do interviews, etc.
• Before you can write a story you have to decide what to write the story about – a “news events”.
• Often the “news events” happen throughout the day and we don’t know what they might be until they happen. Man Bites Dog. Nuclear Annihilation Avoided – Again, Major Caught In Drug Scandal – Again, and so on.
• And so on and so on.

There are a LOT of deadlines in there – and they ALL HAVE MEANING.  As a matter of fact, I’ve left out a lot of details just to keep this post small enough to fit on the Internet.  Also, a number of these things happen “in parallel” – a story can be “gathered”, written about, and edited while other stories are at different stages of “completion” and still all get into the same daily paper.

So I am going to talk about the classic “Printers Deadline”.

There is a HARD DEADLINE in Newspaper Printing

At least where I worked, we had a hard deadline.  To get the paper out to the “marketplace”, the trucks must leave sufficiently early before traffic starts so the papers can be where they need to be when people want to buy them.  That’s the business. So – the hard deadline is: The press must start early enough to start putting newspapers into trucks so they can leave on time.  It is a complex dance with a lot of variables.

To do this, each step along the way has a specific deadline: It’s the time all items for this step MUST BE DONE with enough time to do all the following steps and still be on the press when printing starts.

The bottom line: if a story isn’t ready and “on the press” when the press starts printing, it is NOT going to be in the newspaper (there are ways around this, but that’s the general idea).  That is a hard deadline.  A lot of things can be easily changed right up until that minute.  After that, it is costly in both time and money to make changes.

Sometimes someone will yell “Stop the Presses”, like you see in the movies – but that is rare.  There are occasional “Dewey Defeats Truman” mess-ups.

My point: Real Deadlines Exist

Real deadlines exist. When they do exist, we need a system or method in place to deal with them so we have a reasonable chance for success.  In the newspaper business they have the deadline thing figured out.  The deadline is obvious. Some people say this is where the term “deadline” was invented.  By the way, the newspaper industry has bigger problems than deadlines nowadays, or maybe it’s a different kind of “deadline”, and we can talk about that some other day.

How Newspapers Deal With So Many Deadlines

There are a number of things the newspaper publishers have found work nicely to allow for these serious deadlines.

• Create news features (stories) daily – work in daily iterations – each day, the current “most important” content is created
• Deploy daily – a whole year worth of news is delivered in small, useful, daily chunks
• Decisions are made at the last responsible moment – if a more important story comes in at the “last minute”, a less important story can be removed.
• If insufficient material is generated for a story for it to be interesting or useful, they can quit working on it at any time.
• Whatever is done by the daily deadline is what gets delivered.
• There is no “Yearly” deadline – it’s more like a continuous delivery system.
• They discover processes that allow them to work quickly – e.g.: you can buy some stories from a third party (such as… the AP)
• They can “spice things up” as needed to boost sales.  That is not really important to the point of this article.
• They ca add lines and cut lines from a story to fill the space as needed.
• If a really important story comes in, several people can work on it at the same time.
• Stories that can be worked on, are worked on.  Stories not ready to be worked on aren’t.
• Fillers and other non-news “content” can be done and ready to add as needed without effecting the output of more meaningful content.  For example, a whole weeks worth of comics can be ready each Monday.
• Reporters and photographers can be multi-purpose (they can wear more than one hat).  They can interview someone, take a picture, write a story.  Some can even do the typesetting in smaller houses.
• They only need enough good (or really, “bad”) news material to keep people happy.

News items have different schedules and different deadlines based on the nature of the content – a garden page item might take weeks to write, photograph, illustrate, layout, etc – and a hard news item often has at most about a day and sometimes as little as an hour to go from “this just in” to “press-ready”.  Almost any feature can be edited to reduce its size, increase its size, or rearranged to fit the space on a page.

Sometimes they “stop the presses” and re-set one of the plates to change or add content.  Also sometimes a “special edition” is needed to disseminate late breaking but important news.  Sometimes a flysheet can be printed on another press to add content at the last minute.  Lots of options.

In other words – there are many practices they’ve innovated over the years to be able meet those deadlines and do the job of delivering a sufficiently interesting and sell-able product.

Sometimes Deadlines Are Needed, and We Need Ways To Handle That

Well – this blog is about software development, not the newspaper business.  However, I think there are a few similarities.  What do you think?  You probably see other similarities, and a few differences between the newspaper business and computer programming.  Still, the newspaper model might have some use.

Do you have deadlines??? Learn how to deal with it.

Cory House ( blog  , twitter) wrote a comment on my Estimate Chess post, and I’d like to double purpose it by turning it into a post.

First – thanks for reading my post, Cory, and for making comments.  I appreciate that very much!!!

Cory’s Comment

I can appreciate the feeling of fruitlessness in some estimates. It’s hard to get right, particularly when requirements are weak. However, I believe a professional developer must be willing to take the risk and do the hard work of estimating when requested. Drawing a hard line of “no estimating” without considering context doesn’t come across professional or accommodating.

Your chess example is entertaining, but it’s also a great example of a place where I feel it’s unfair to expect an estimate. Given, it’s gray exactly how your story would translate to the development world, but it seems to describe a request to estimate a project without even vaguely reasonable requirements. I think we all agree that’s a waste of time.

If given sufficient requirements, we, as professionals, should be able to estimate our work. The world expects no less from engineers in other fields. Multiple contractors recently estimated the cost to finish my basement despite the fact that I had no floor plan, no idea where lights would be, only a vague idea on what word work I’d like, a bathroom of unknown size and quality, etc. If one of them refused to provide an estimate, I’d have gone elsewhere.

I land plenty of dev work on contract because I’ve gotten good at asking the right questions and sniffing out complexity so that I can provide reasonable estimates. Being skilled in estimation sets you apart whether you’re independent or full-time. Steve McConnell wrote an excellent book on the subject. http://www.amazon.com/Software-Estimation-Demystifying-Practices-Microsoft/dp/0735605351   That said, while I disagree with your answer to “What can we do?” as “Nothing.”, I enjoyed reading your blog. It’s fun reading and certainly food for thought.

My Response:

Hello Cory! Thanks for commenting on my post!

Well, as you can guess, I don’t agree with your take on things – but that’s why I have a blog, and you have a blog. We all get to write about things as we see them.

Regarding professionalism: I believe it is professional to find a better way to do all the things we do, and eliminate things that are wasteful. I see estimates and the estimating process as mostly waste, and as a professional I see no need to “take on the hard work of estimating when requested”. More directly: estimating is not hard to do, in my opinion. I’m not clear on why you might think it is. However, I also believe that estimates as used in software development typically (and almost always) result in misinformation that is used to make bad decisions. Which is why I follow an Agile approach to development. The Agile approach, at least as I do it, provides a lot of value and eliminates a lot of waste. And that is very professional. We’ll just have to agree to disagree on our thinking about professionalism and estimates.

I don’t think I made it clear in the post: The Chess analogy is to show that something as simple as Chess becomes complicated quickly. It is a VERY SIMPLE GAME, with very few variables, and yet it becomes too difficult to predict when we add a few simple constraints. And even without those constraints, there is little we can predict. Otherwise we wouldn’t even play the game, right? Remember: Don’t stretch an analogy too far – it will always break. With software projects, anything that is non-trivial is much more complicated than the simple chess example.

Cory: “If given sufficient requirements, we, as professionals, should be able to estimate our work”.

My Response: This is where we really diverge. I see estimates as waste, in the Lean sense of the word, and most other ways. Estimates are not part of what the customer buys. They don’t add value to the product. In a value stream map estimates are on the “non-value” track. They are merely ONE WAY people have attempted to be able to arrange working relationships and get software created. It makes sense, and it partly works – still, there is nothing about them that makes them necessary, or even beneficial to the actual creation of software. We can write software without them. We can do whole projects without them. And if we can find a way to make BETTER SOFTWARE at less cost, and to more quickly get the right software into use without estimates, wouldn’t it be professional and worthwhile to at least consider doing it that way? Hopefully we can agree on that.

Cory: “The world expects no less from engineers in other fields. Multiple contractors recently estimated the cost to finish my basement despite the fact that I had no floor plan, no idea where lights would be, only a vague idea on what word work I’d like, a bathroom of unknown size and quality, etc. If one of them refused to provide an estimate, I’d have gone elsewhere.”

My Response: Here we continue to think differently. There are countless examples in life where estimates work to some degree. We’ll discuss that some other day – most, if not all of them are NOT useful as models for software development. Here is where I’d like to leave it for the moment: If you could have had the work done on your basement for less money and get higher quality, and perhaps get it done even quicker – would you be willing to consider doing that? I know that there would be no way to know it was cheaper if you had no estimates – but conceptually: If you could get all the benefits I describe would you consider learning to do business that way? I hope we can agree that would be better all other things being equal.

We have contracts, and estimates, and predictions, and all the trappings of doing business this way because most people haven’t found a way to do things better – we fail each other, we don’t deliver as expected, we don’t trust each other, so we find ways to do business that gives us a chance to do business. So… we can try to get good at estimates, and it if serves you and your customers, more power to you.

Lastly:

Cory: “That said, while I disagree with your answer to “What can we do?” as “Nothing.”, I enjoyed reading your blog. It’s fun reading and certainly food for thought.”

My Response: I didn’t ask “What can we do?”. I asked “What can we do instead of doing estimates?”, and the answer was simply put as “Nothing”. My point being that we do not need to do anything “instead” – estimates are often simply UNNEEDED. And then I posed the more meaningful question: “How can we develop software without doing estimates?”. The nuances of this are important, and perhaps I was being too cryptic. At the time I wrote this post it seemed few were willing to explore these things. I hope I was wrong, and now I am certainly finding more people willing to entertain and even experiment with finding better ways. And some doing purely “no estimate” projects.

Overall, I have found the Agile way works well. When you have customers who see the benefits and find it works better for them than the old phased, predictive approach – then it is truly sweet.

Thanks for your kind words. I try to be fun, and I’m “all about” food for thought!!!

Cheers!

I’ve done 12+ sessions at Agile and Developer “user groups”, Agile or Dev conferences, college classes, and code camps where the main topic (or a big component) of the session was Estimates in Software Development.

Here are the notes from one such session I facilitated last year at an Open Space event (Agile Open California, North)

Session Title: Estimates – Can’t Live With Them, Can We Live Without Them?

The title was my own.  I’ve been talking about, and holding sessions on this subject for several years, and the previous time I was at this event a couple years earlier I did a session on “5 Whys on Why we Estimate”.  It was a very contentious session, with almost everyone (except me) feeling estimates are important and needed for software development, and the problem is just that we’re no good at it… so we just need to get better at doing estimates.  I don’t agree (as you probably know).

The “we just need to get better at it” argument sounds a lot like what I used to hear in “post-mortem” and “lessons learned” sessions in waterfall/phased project environments. A typical “take-away” from those sessions were things like this: “We missed our deadlines and had a lot of rework because the requirents kept changing, even though we have a change control process in place.  We need to get better at controlling changes to requirements”.  I hope you see that is very likely a bad conclusion.  Essentially, that line of thinking is that “since control isn’t working, we need more control.”

I’ve been suggesting that if we keep failing at getting good at estimates even though we are working hard at becoming better at doing so (and have been doing so for countless years), then maybe we are expecting estimates to do something for us that just can’t be done.   Maybe there are better ways.  [Just to be clear, for me – I feel I have found a much better way and it is Agile Software Developent as presented in the Agile Manifesto (Values and Principles) – the Agile MVP.]

My position: We need to get to the bottom of what it is we want from estimates and predictions, and find a better way to get to what we really want. [HINT: What we reallly want is Sucessful Software Development Efforts (or something like that, perhaps?) .  Or, as long as we are wishing, let’s just wish to win the Mega Lottery and then spend our day’s doing something we really enjoy. Like taking hikes or arguing – whatever you like.]

Participants:

There were initially 17 participants.  Developers, ScrumMasters, Agile Coaches, Product Managers, etc.  Almost everyone had some experience with Agile/Scrum/XP/Lean.  A few people joined in as the session proceeded, and a few others left.

Purpose:

Explore and discuss what we want out of estimates and the problems we’re having with estimates.

[NOTE: Everything will be tainted because Woody Zuill is facilitating this session, and he leans slightly toward the “No Estimates” approach.]

Process:

My approach for this session: Quickly explain my basic thoughts about the failure of estimates and the purpose of the session, do an information gathering exercise.  Ask why we feel we need estimates, have the participants write their thoughts on sticky notes, group them, and then discuss.

Information Gathering: Why do we “need” estimates

• We gathered sticky notes to identify “why we ‘need’ estimates and put them on easel sheets
• We then gathered the notes into groupings.
• There were 2 main groupings: Estimates are for planning, and estimates are for manipulating
• NOTE: Tne notes below were written by the participants.  Some are a bit cryptic. Not to worry, okay?

• Here are the notes and the groupings:

Affinity Group: Estimates are for Planning

• How many resources do we allocate to do the work
• Length of engagement
• Costs
• To Predict Future Velocity
• You need estimates to determine budget
• To determine when we’ll finish
• Target implementation time-frame & duration for resource allocation & cost estimates
• When to commit to customer delivery
• to decide whether or not to do a task
• To Determine when a feature will be available
• To determine when we’ll finish
• To know when it is done
• It drives the business
• So we can prioritize
• Business needs it for planning
• Request funds for work
• To help balance the work
• Helps in managing resources
• Determine if end result is worth the cost
• To give the customer prediction – to meet customer expectations
• CEO needs it.

Affinity Group: “Gaming” or manipulating the system and related stuff / dysfunctions

• NOTE: I am not discussing these things in this post – sometime soon I’ll cover this stuff in detail.
• The keep your job
• To give a reason to yell at someone
• It’s “How we do things”
• Habit
• Need to get a programmer fired (setting her up for failure)
• To cover your own ass as a developer
• To pit people/teams against each other

Now for the discussion

After our gathering and grouping exercise, we began discussing what we felt this showed us.  During the discussion we just “followed the flow” as ideas were expressed.

We observed, as a group, that estimates are mostly for planning and making decisions about what to work on.

In the discussion, we were in alignment that we “need” estimates so we can plan our work, and so we can make decisions about which projects or features to work on.

After all the effort we put into estimating, we still rarely get things done “on schedule” and “within budget”.

Also, we still frequently ask (or our managers ask): “Are we there yet?”.

Why do we have these problems?  We’ve been doing estimates for years – shouldn’t we be great at it by now? Other similar or related things that we’ve seen:

• Progress isn’t visible
• We seem to be get more behind our schedule no matter what we do
• Money is running out and important things are not even close to being done.
• Stuff we thought was done turns out to not be.

Rarely are the estimates accurate enough to really count on for release dates, costs, etc.

How do we typically deal with “over budget” and “not on schedule” issues?

• We end up cutting features
• Pushing out deadlines and so on
• Hire more people
• Blaming each other about whose “fault” it is
• Agreeing to get “better” at gathering requirements and doing estimates “next time”

Still, others are dependent on estimates such as marketing, HR, Ops, our customers… and so on

• Do estimates really provide sufficient information for these folks to get real value?
• Or… are there a lot of decisions made based on faulty estimates???

So, With all these problems, why aren’t we looking for a better approach? What is wrong with our estimating process? Should we try to get better at it, or… ?”

One reason estimates provide little chance for usefulness:

• The Unknowns might invalidate any estimate we make – estimates by nature contain unknowns
• What is 4 x 8 + 7  / 2 * 83 + unknown?
  • The answer UNKNOWN
  • We can guess, or base our UNKNOWN on similar things.  Might work. Might not.
• Many important decisions are based on UNKNOWN, but we act as if we “KNOW” something
• The argument is often made that we just need to “have some relative estimate…”
  • It is likely that even this leads to bad decisions
  • For example, what good is relative decision based on “This UNKNOWN is bigger than that UNKNOWN?”

MY CONCLUSION AS PRESENTED:

So, if the Estimating process is flawed (and it might be), and getting “better at it” hasn’t worked… Do we really need estimates?  Can we innovate a better way?

• What I am suggesting is that we have to evaluate our dependence on estimates
• My own experiences have proven to me that estimates are not always needed.
  • I’ve been working without estimates for over 3 years with good results in corporate, “internal” projects
  • I’ve had several long term ongoing projects that did not use (or need) estimates – One of over 4 years on commercial software
  • I’ve done contract work with NO ESTIMATES. Many have told me “customers won’t do that”, but there are at least some that will.
    • When a customer sees that more “real” work gets done and more of the right thing gets done with less “busy work” they get on board.
    • Finding ways to pay a very low cost to prove something works, or doesn’t work is very attractive to most customers.  To me, this is a fundamental benefit of an Agile approach.
• My experiences are not proof that estimates are not needed.
  • I am not trying to change the world.  I do want MY jobs/contracts/customers to get the benefit of doing things in a better way.
  • I AM trying to invite conversation about this stuff, and encourage us to explore and replace these painful practices in our profession.

This discussion had more to do with posing the questions than answering them

• It is contextual (as usual)
• Lots of organizations feel they need estimates and you must work within the system.  Sorry for that – but I can’t fix that.
• Often, customers will not entertain using a company that does not provide an estimate.
  • They want to know what they’ll get, when they’ll get it, and how much it will cost.
  • It is a risky situation for everyone – and if you are in that world, there are still ways “sell” the benefits of finding a better way.
• I am suggesting is that we need to do some serious thinking about how to change this reality, and innovate ways to eliminate the dependence on estimates

I am proposing that we need to imagine a different world

• Do some “thought” experiments
  • IMAGINE:
    • What if we were to find that estimates have NO value (zero benefit), should we still do them?
      • What would that look like?
    • What if we were to find that estimates are harmful and are counter-productive?
      • What would you do?
    • What if the were to find out that estimates are fatally TOXIC
      • What would we do then?  How fast would we need to address this?

Take some action that will expose the reality about our estimates

• Can we find a way to wean ourselves from estimate dependency?
• A few things that have worked for me
  • Deliver value quickly and frequently (sound familiar?) – that is: put small chunks into real use early and often
    • Making decisions about “what to do” become less important when we can do “some of this” then “some of that” and see what works (or doesn’t work) quickly and cheaply.
    • People stop worrying about “are we there yet” when interesting (useful) things are continuously happening and being put into real use by real users.
      • It’s like bringing along some new toys and activities to hand to your kids on long trips when they start getting antsy.
  • Lead your customer/stakeholders/whatever to an understanding of what is important to deliver NEXT…
    • … we only have so much work that can be done right now – focus on that
    • … put all your effort into that
    • … and put little time on all else
  • You can introduce this a little at a time.
    • Thinking people who are open to this will slowly come around.  Or not.  Just depends.

Remember: I am just encouraging discussion.

What works for me might not make sense to you.  Maybe someday it will.  When that happens, you will have crossed over into the land of no estimates.  Beware.