Is there a link between estimating and psychology? I certainly believe that there is – especially when it comes to software development. As humans, we react to what others tell us. Sometimes the results are surprisingly positive and equally likely the results may be negative. How we react can give us insight into how others might react – and why.
When estimating the duration or effort of any activity, the word estimate means: (from freedictionary.com):
es·ti·mate (st-mt)tr.v. es·ti·mat·ed, es·ti·mat·ing, es·ti·mates1. To calculate approximately (the amount, extent, magnitude, position, or value of something).2. To form an opinion about; evaluate: “While an author is yet living we estimate his powers by his worst performance” (Samuel Johnson).n. (-mt)1. The act of evaluating or appraising.2. A tentative evaluation or rough calculation, as of worth, quantity, or size.3. A statement of the approximate cost of work to be done, such as a building project or car repairs.4. A judgment based on one’s impressions; an opinion.
Note that it does not mean prediction or forecast — but rather a rough calculation.
But, here is what happens when someone gives us an estimate: our minds access our historical database of what happened with a similar event in the past and gauges whether the “estimate” has a likelihood of being on time, early or late. It does not matter if the person(s) involved in the current estimate had anything to do with the past because our minds will find similarity and tie the two events together. If we have no memory of a similar event, then we try to tie the estimate to our experience with the estimator or team. In the case of no similar event or persons, we simply go on blind faith that the estimate will be correct. If the estimate is accompanied by an estimated range of accuracy (e.g., plus or minus a % or timeframe) it “feels” even more reliable, when in fact the estimate is still only a best guess (albeit educated) of when an activity will start or finish, and we often expressly accept it.
Then if the activity finishes early (in the case of software estimating we assume customers will be pleased) we are generally surprised, pleased or irritated depending on our own plans (i.e., if we made plans based on an estimate, we might be irritated with an early delivery. On the other hand, if we hoped for an early finish, we might be surprised and pleased.) The same holds true for others. If a program or function is delivered early and we rely on our customers to test it, before we can move forward, our customer may not be ready in time because the delivery exceeded the estimate, or they may be elated because their need for the product was urgent and you over-delivered.
When the delivery is “on-time”, it simply means that life went according to plan per the estimate: durations were right and no unforeseen life events came into play that prevented the on-time delivery. When this happens, our trust in the estimator and the team increases, even when it may not be warranted. (The estimate may not have included contingency for life events and the estimator was simply lucky that things went according to plan.)
When the delivery is “late”, it simply means that life intervened more than anticipated due to increased effort or duration (“we didn’t anticipate that this would be so difficult”). Our reactions to this lateness of others can range from the mundane (“typical – I knew they were too optimistic”) to anger (“now it’s going to cost me more”) to relief (“thank goodness they are late, now I can also be late”). Additional psychology happens with late deliveries and we judge (and store in our memory) future estimates on the spot. Our customers do the same thing – they judge our ability to deliver when we are late – even if they do not understand how their own role or lack of participation contributed to the lateness.
In software estimating, an interesting disconnect occurs on the part of corporate memory. When a supplier provides an estimate of the work it takes to do a phase or activity, the estimate is often perceived as too high (too costly) and management will routinely cut the estimate in half or less and expect an on-time delivery. More often than not, the estimate was exactly the opposite and was overly optimistic on the part of the delivery dates. As a result, more often than not, the delivery ends up being late for several reasons: 1. the new estimate (less than the first estimate due to management gut-feel that the original estimate was high) was unrealistic; 2. The first estimate is accepted but was overly optimistic; 3. The estimate was based on an artificial delivery date set before the project was even launched (before the scope was known) and the date became immovable once it was announced; or 4. Life intervened and what was anticipated for the project was not what was planned. If no contingency for anticipated and unanticipated factors was included in the estimate, the delivery will always be late (unless less is delivered).
Does it not seem obvious that an unrealistically low estimate will predicate a late delivery? Yet, the situation repeats itself in IT reminding us of the wisdom of Einstein: Insanity is doing the same thing repeatedly expecting different results. To fix the dysfunction in IT of slashed estimates and late delivery, customer sponsors need to overcome their urge to adjust the estimates (no matter how high they may seem to be), and suppliers need to include ranges around estimate accuracy. As such, an on-time delivery becomes a realistic possibility.
When our minds react to dates – even unrealistic ones – we set up a basis for future reactions. Surprise is not a comfortable feeling – even if it is part of a positive result, and so our minds will entrench a historical reminder to prevent such reaction in the future: “the next project will also be early” or “they will never get it right and will always be late” or “they were lucky to get finished on time” are potential responses.
An example of early, uncomfortable delivery happened to me this morning. I had to catch a 7am flight from Dallas to Washington, DC and had called last night at 10 pm to arrange an early morning shuttle pickup at my hotel. The service told me that I would be picked up between 4:35 and 4:50 am so I set my alarm accordingly. I judged that the service was going to be very accurate because: 1. It was within 5 hours of my pickup time; 2. they provided a 15 minute window (and my history with the service was that they would be late); and 3. I talked to a real person. Imagine my surprise (and displeasure) when I received an automated phone call at 4:00 am alerting me that my shuttle was within 10 minutes of arriving. Less than five minutes later, another phone call informed me that the shuttle was now waiting at the door and any delay on my part would keep others waiting. This was a mismatch with my experience (shuttles would run up to an hour late) and it interfered with my plans to be on time with their original estimate. Had the service told me that they’d be there at 4:30 am plus or minus a half-hour, I would have been better prepared, and without an explanation, I can only assume the early delivery was for their convenience in arranging the least number of trips (two others were in the van already and probably had earlier flights to catch).
Using our own life experiences prepares us to face the realities of estimating and the psychology of how others react to on time, early and late deliveries. This helps to understand why our anticipation of user delight over early delivery may not be met, and how our perception of estimates may not be shared by those with downstream involvement on our projects.
To your successful projects!