Overpromising and Underdelivering

Background

A lot of topics have run through my head the last few years as I’ve mulled over the eventuality of this blog. They’re still there. A good chunk of them are in a list on paper. This, then, seems like a good place to start. There’s no one I overpromise and underdeliver to more than myself.

At one point, this tendency was an asset. As an entry-level professional from whom time estimates were ignored, being wrong on what can be accomplished in a given amount of time reflected well on eagerness moreso than it did poorly on understanding. The folks bearing the brunt of my exhuberant optimism were peers and immediate managers, and they learned pretty quickly to do their own mental math on my work estimates. It’s also an expression of a side of my personality that I don’t want to give up on–I’m an unabashed eternal optimist.

Eventually though, the other edge of the sword brought itself to bear. Friends and lovers tend not to appreciate words not backed by action. At work, trust in my input and experience has grown, and that includes trust in my estimates. When those estimates aren’t met, or are met with a subset of the promised functionality, that’s no longer a demonstration of naive excitement. At best, it’s an annoyance and at worst it results in missed deliverables.

I’m getting a little too far into the vacuous weeds, and that means we’re past-due for story time. A little over a year ago in my 9-5 we inked a deal for an embeddable BI product that would let us move report-writing labor from the development team to accounts management (and in some distant future our clients themselves). I got real excited with our evaluation phase, estimating a ~month to get the frontend integration into devs' hands and missing no opportunity to point out how this project would be a great time to proof-of-concept some improvements to every layer of the stack. By the time I got the product in the devs' hands (much less accounts folks') eight months(!) later, it became apparent it was a poor fit for what we needed from a BI solution, we declared it a sunk cost and moved on to an entirely different solution.

Here’s what I should’ve done¹:

• Used the existing vendor API I stood up during the evaluation phase, and immediately written a toy example that integrated the client code into a demo app for our very-green devs to work off.
• After getting the fuck out of their way, iterated on the litany of infrastructure improvements needed for this to be a robust part of our product, taking the opportunity with each improvement to work closely with a dev to get them up to speed on how they can use that improvement to ship more useful features more frequently and reliably.
• Talked less specifically about how wonderful this integration would be for our products and broader dev workflow.

Here’s what I did:

• Spent ~2 weeks architecting this new infrastructure and glue code, plus the most likely iterative improvements for the next next three sprints to get to the land of strawberries and orgasms that I’d promised.
• Wrestled with the broken corners of this product’s API to shove its old-school .NET circle into the square hole of immutable infrastructure as code.
• Wrestled with Terraform’s woeful ECS workflow to deploy ETL code.
  • Realized this is a textbook Lambda usecase, and spent as much if not more time writing Terraform-as-deployment-DSL for any Lambda as I did with the actual ETL code.
  • Attempted to do the crosstraining in bullet 2 in the above section for some of our pre-existing ETL. Since tabled.
• Threw the initial client code proof-of-concept over the wall for a junior dev to implement on top of an already-fulltime workload.
  • Subsequently wrestled with idiosyncrasies in the vendor’s API documentation in an attempt to get the client/embedded integration proof-of-concepted.
• Treated all of the above as more or less concurrent things that the MVP depended on, and bounced between them when any of them got too frustrating.
• Understood that this Hindenberg of a project was floating on sunk cost at about T+2months and continued giving overly-optimistic estimates for the next six.

This is one example of many both professional and personal, but depth is probably more effective than breadth to demonstrate how this shakes out in a deeply unhelpful way. I need a new behavioral paradigm, an alternative that doesn’t suppress demonstrations of that eternal optimism I so enjoy about myself. Let’s review some of the options.

The Scotty Principle

We unrestrained optimists owe Star Trek for the most common suggested correction to overpromising: the Scotty Factor, or (my preference) Scotty Principle. Usually the suggestion goes something like this: take the time estimate you would’ve given, multiply it by four, and give that. That’s an alright antidote to overpromises revolving around time (though not without its own flaws), but promises relating to time are only a subset of the problematic commitments I make on a regular basis.

At least as often as I miss promised dates, I miss on some or all of what I promised to deliver. As self-imposed crunch hits and the need to deliver something bears down, I remember the practice I should’ve been following from the beginning and start shipping smaller subsets of the promised functionality in a more iterative way. But because I stall the implementation process too long in pursuit of the perfect design to match my lofty self-imposed requirements, I wind up shipping the MVP at T+2months instead of T+2weeks.

Enter #NoEstimates

I need a solution that accounts for both misestimates of time and scope. A more recent parallel methodology, the #NoEstimates movement, seems a perfect fit. I should be clear that while the folks that write about this write a lot about agile software development methodology, under the covers it’s mostly about the benefits of cost-plus contracts, especially in contrast to one alternative. Some subset of folks writing #NoEstimates content might disagree with this characterization. I’d challenge them to look at the audience engaging with this purported movement. Refusing to acknowledge that it’s about money in favor of vague/nebulous/easily-gamed Agile™ story points² or whatever you call measures of contribution magnitude when evaluating individual KPI is being deliberately dense³. If you’re a click away from @ing me, don’t. It doesn’t matter, and I don’t even care.

That took an irreverant and antagonistic turn⁴, but reality is undeniable here: at some point, someone pays you to deliver code, and even in the cost-plus model some informal estimation (if only that implied from past behavior) must occur, or the person footing the bill won’t open their wallet. If you want to read five hundred words that say more or less that with more nuance and a bunch more context, go read that link to Fowler’s blog.

All that said, any approach that values behavior and delivery over words and promises is a good one in my book. I just think I have a better way.

My⁵ Scotty Principle - A Paradox

This better way is an adaptation from a core behavioral therapy method, and it boils down to essentially two practices:

• documenting the promises I make when I make them
• returning to the document to note whether that promise was fulfilled

Since there are two dimensions to meeting the promises I make, I’m approaching it with a five column table that looks something like this:

The closer I match this table to the reality my commitments create, the more vigilant I am in returning to the table to note what I actually deliver, the less I run my mouth and the less I have to note. Not really a paradox, but I never claimed to be Zeno either. Like infinite divisibility, this exercise in note-taking never really ends. So I’ve got that going for me, which is nice.

My Promise to Myself - The Paradox At Work

Note the second-to-last line in the table above. Committing to this exercise is itself a promise to myself, and self-harm might be the greatest harm inflicted by this maladaptive expression of optimism. I need a way to measure both how well I’m meeting this promise to document my promises, and whether it’s having the intended effect.

Delivery-side Over/Under

For simplicity’s sake, I’m going to add a tally every day that I track a promise in this list. Even if the hoped-for result (running my mouth less) comes to pass, I don’t foresee a world where I go through a day without making a commitment. Even if that commitment is just to get back to the other party when I have enough information to make a committment.

The exercise in introspection that has been writing this first(!) post for my blog over the course of the last four(!) months has left me under no delusion that I’ll get 100% here. But it seems fair to say that if I don’t meet this goal 70% or more of the time the below evaluation is more or less meaningless.

Outcome-side Over/Under

I need a better way to measure my hypothesis here, but I think in the medium term (~T+1 year) I’ll make it a weekly reflection to tally up the times I meet my ETA and delivery goals versus the times I don’t. I don’t have a good level of self-awareness to ballpark this, and I’ll just be happy if the former increases while the latter decreases in both cases.

tl;dr

Overpromising and underdelivering sucks for all involved. The way out isn’t some seemingly-pithy formula, nor is it a highly opinionated, reductive hashtag. It’s tracking the promises I make after I make them and subsequent presence of mind the next time I’m called to make one.