An unbroken chain of little waterfalls

The Buffalo River, nestled in the Boston Mountains of Arkansas, is one of the best floats in the country. It's so iconic that it's been protected as a National River, the first of its kind. Clear water, tinting to green-blue in the depths. White limestone rocks and pebbles studded with the fossil remnants of a shallow Paleozoic sea. Gentle rapids that cascade your boat downriver, each one a little waterfall that is so smooth your canoe never feels out of your control. A float from the put-in at Tyler Bend to the Gilbert General Store takes about 4 hours if you're looking to enjoy yourself along the way.

But this isn't a post about floating down a river. It's a post about Agile, Waterfall, and the challenge of estimating time and complexity as a Product and Engineering leader. But it's also a little bit about rivers, journeys, and Flow.

There are countless ways software and product teams use to estimate how long it will take them to ship a feature or complete a project, precisely because all of them are so bad. I suppose Point Poker sells those silly card decks, so it makes someone money. But Fibonacci points, T-shirt sizes, point poker, time estimates, and all the other idiosyncratic things people resort to under pressure to perform better at estimation than the last late-shipment are well... pointless.

If you ship consistently on time, when has your point-poker (or whatever) exercise ever told you something you didn't already guess intuitively? If you ship consistently late or early, and you go over your estimations every time, when do you get anything but "reasonable excuses that we couldn't have known better" for why the estimate was so far off?

There's a book, my old colleague and mentor James Coffos at Teamworks gave to me when I was trying to ship our re-engineered product stack on time. Actionable Agile Metrics for Predictability: An Introduction, by Daniel S. Vicanti. Despite its longwinded and dull title, this is probably the best, shortest book I've ever read on figuring out how to recognize speedups and slowdowns, how to estimate correctly, and how to intervene when the float down the river from concept to feature snags on the shoreline.

First off, humans are tremendously bad at estimating time and complexity, and there's no evidence they can get much better when the approach is "narrative guessing," i.e. reading a description of a feature or ticket and giving an estimation based on your understanding of what's written.

It's far better to estimate completion of a task based on past performance. Start with a well-written epic of fixed scope. Break it out into tickets with engineers and allow those tickets to be broken into subtasks. (I'll tell you in a minute how to do all that.) Then, at the end of each sprint measure:

• The number of tickets and subtasks completed.
• The number of changes to the scope of a story or epic e.g. the end-goal, requirements, and user-stories.
• The number of new tickets and subtasks added.

By "epic of fixed scope" I don't mean that the tickets are static. They can be added to and designs can be reworked, but the outcome should remain steady.

Over time you're going to build a picture of what a good project looks like and what a troubled one looks like. From these measurements above you want to understand how fast on average your team moves through tickets vs. the amount of "scope creep" and "unexpected complexity" they discover per sprint. You won't believe me until you measure it for a while, but regardless of how they estimate it, your teams are going to move through tickets at roughly the same rate every month.

The canoe-crushing boulders on your project are not velocity, but scope change, creep, and undiscovered complexity. There's some wiggle room on these rules, but:

• Changes to the end-goal or core requirements of a story or epic should be done by the time devs accept it and begin work.
• From there, tickets and subtasks should stop being added within a sprint or two.

If this doesn't happen then the feature definition is incomplete or the devs lack clarity on how to build what's being asked. Further attempts to develop against it without refinement will result in wasted work. Schedule a retro and re-evaluate the scope and tasks before going forward again.

If this sounds like Waterfall to you, understand that you cannot deliver reliably if you don't know what you're building. I'm not saying that you build a whole new product with waterfall process. I am saying that the most agile way to develop is to navigate a chain of tiny waterfalls that become clearer as they're approached. Rapids, if you will. An epic at a time.

This of course puts a hard outline around how an epic can be constructed. It has to describe a precisely known modification. It covers maybe six weeks, not six months of work. It also can't be a title and a bunch of stories which themselves are title-only. It has to have written, fixed(-ish) requirements and end-goals when it's accepted as dev-ready. If you're building something bigger, it's more of an "initiative" and covers multiple well-understood modifications to achieve a larger goal in an agile way.

What's a well written epic look like?

A well written epic distinguishes between the Product function and the Engineering and Design functions. Typically, I think of product concerns as user stories, acceptance requirements, stretch goals, milestones, and releases. Organize what's being built in the most declarative (as opposed to procedural) terms possible. You want to give your engineers and designers freedom to come up with a solution that fits the experience of using the product, and you can't do that if you're telling them how to do it.

Figure out the who and focus on the outcome they want.

I'm going to go with an example. I don't want to rehash how to do basic product research but your decently written epic is going to focus on a class of users and a problem they want to solve.

First, an example of a well-written epic

Feature: Scheduling time between Personal Fitness Coaches and their Clients.

Need to have:
* PF Coaches need to configure regular blocks of time in 15 minute minimum increments.
* Recognition of existing calendar commitments, both scheduled blocks and other appointments and time-off on the PF Coach's calendar.
* PF Coaches need to share the block list with clients.
* Clients need to be able to use the block-list to schedule an appointment without direct human-to-human interaction.
* Clients need to be able to customize the subject of the appointment.
* Clients need to receive a confirmation message with a link to cancel or reschedule the appointment.

Nice to have:
* PF Coaches would like to be able to automatically close time after a booked appointment in order to write up notes and followup plans.
* PF Coaches would like to be able to attach videos and files of exercises to help their clients prepare for a session.

Eventually need:
* PF Coaches need a way to block off training equipment or facilities without having to open a separate calendar.
* Clients need to receive a text message the day before their appointment (with the ability to opt-out) reminding them of their appointment, and offering a last-minute cancellation or reschedule option.

This isn't the only way to write an epic, but it mirrors how entrepreneurs think. There are need-to-haves and nice-to-haves, and follow-ons that they deem natural, but aren't included in the ACs. Each bulleted "user story" or AC is written as an outcome that a class of user needs to achieve using your software.

Designers and Engineers should talk with the person guiding the product definition (could be a PM, could be the CEO, could be the Founding Engineer) and clarify the requirements and fill in gaps before going off and creating designs and engineering artifacts. For example, missing in the above but probably necessary is "PF Coaches need to be able to cancel or reschedule an appointment individually or for a block of time / days." and "PF Coaches need to be notified if a conflict arises with a client from an incoming appointment on their calendar." A good designer or engineer will catch that requirement as missing and add it.

Designers will take these ACs and turn them into full fledged user stories with screen or dialog designs. Engineers will say "oh yeah, we already have a recurrence dialog, so don't design a new one" and debate with the designers on how to get into the scheduler in a natural way. Then they come back with the person guiding the product definition and go over their approach. That approach should be complete in the sense that it covers the project from breaking ground through delivery. It's not just the first sprint's worth of tasks but an outline of how the project should go. Sure, more will get added along the way, but Design and Engineering should know how they're getting there.

Also, If the product person takes the design to a customer call or two, the lead designer and engineer on the team should be present on that call, because they're the ones that need to get better over time at intuiting how a customer wants to use their software. Once everyone agrees that the solution is workable, it's tasked out:

Engineering Tasks:
* Scaffolding: Create a microservice for the block scheduling API
* API Command: Create Block Schedule using an RRULE
* API Command: Book appointment
* API Command: Cancel appointment
* Subscription: Block Scheduler should respond to incoming calendar appointment changes.
* API query: Client view of a week of free time in the block schedule
* API query: PF Coach view of a week of free and booked time in the block schedule.
* UI Component: Display summary view of days that have free blocks.
* UI Component: Single day booking view.
* UI: Transition from summary view to a single day's booking screen, like Calendly's workflow.
* Publication: Use the messaging service to send a confirmation on appointment booking
* Scheduled Action: Use the messaging service to send a reminder at a configurable, fixed amount of time before the appointment.

And so on. If you're thinking "This looks like a lot of work before engineers start building my product," then one of two things is true:

1. You're a tiny shop where everyone knows what they need to build and this is too heavy-handed a practice. This article can wait until you're bigger.
2. Consider this up-front cost vs. the cost of wasting engineering and design days and weeks needing to redo work based on unforeseen changes.

I cannot tell you the number of times I've seen "seat-of-our-pants Agile" result in dead features or weeks or months of being stuck at "80% done" on a project. If the person doing the product definition is incentivized to commit to the work they're pushing towards engineering and product, and they're held accountable for late-changes, that person is going to get better at their job quickly. If the engineering and design functions are given creative control over a solution, then they're ultimately held accountable for coming up with a good solution for customers, making them more market and customer focused.

When the above is practiced well, it encourages Flow in your team. Product is incentivized correctly to keep the process moving. Design and Engineering are given the ability to work to their strengths and become better at understanding your market and customers.

Now for some Bad ones

Here are some "anti-patterns" I've seen that cause projects to drag out.

The One Liner:

Slots in the Calendar for Personal Fitness Coaches and Clients.

This makes the design and engineering functions guess at what the product is. The worst part of a one-liner is that it's usually deceptively straightforward. It implies that the product person thinks that the result is obvious enough that engineering and design should just know what to build. Realistically they're going to think of about 40% of the actual ACs. The "stories" that get added will be a mishmash of engineering tasks, design artifacts, and guesses about what the user wants.

The result will be that the PM goes back and forth with customers with the half-finished feature and saying "we actually need this change," to the building team. If it was a 6-week project done right, it's now a 12-week project that creates tension and bad blood between engineers, designers, product, and customers.

Product Negotiates Against Itself:

If the product manager is not also the engineer and the designer, then they do not know what constitutes complexity for those functions. If they don't understand this limitation, the temptation is to "offer" engineering and design "concessions" that actually fail to reduce complexity at all and in many cases increase it and make for a worse customer experience at the same time. For our above example, these kinds of ACs get added by product management before the feature ever reaches an engineer or designer:

* Coaches should only see monday-friday, 9-5, for their schedules
* Clients have to log in to see the scheduling page.
* Confirmations are SMS only, no push or email.

From the product manager's perspective, they've reduced the number of days and the time window that need to be considered, made it so you don't have to handle anonymous scheduling, and you've reduced the number of channels a message has to go through. From Design and Engineering's point of view, these are requirements, not concessions and well, now:

• We can't use our existing calendar components because of the restricted schedule.
• The sign-up flow has to account for a new user sign up happening during the scheduling workflow. Redirects and navigation need to be updated in the mobile and web apps. The user's experience is made significantly worse because they have to complete an irrelevant task in the course of doing the thing they came to do.
• Our message provider Twilio already provides SMS, push, and email as a single notification package so now we write code into the system that allows us to ignore the user's notification preferences and only send SMS. Now the user is irritated because every other message from your app comes according to their preference but this one, new, "buggy" feature they used.

Instead of cutting scope, the product manager just doubled it and created a series of follow-on tickets that will get "mopped up" after the initial release when the coaches complain that they can't do things like schedule evening appointments, and when clients demand a "lite" experience that doesn't require downloading the app.

Prescriptive instead of Descriptive:

Here we have no user-stories. We have a prescription by Product of what they want built. It might come in the form of a slide deck or Figma they sketched designs on. It might come in the form aberrant "user stories" that are actually written as ersatz engineering and design tasks. But however the work is presented, the product manager has done Engineering and Design's jobs for them albeit badly, setting up a chain of painful and slow negotiations where Design and Engineering suggest changes without understanding the underlying desires of the user they're building for.

Now your builders are building for the product manager rather than the customer. The end-product will inevitably be incomplete and buggy, because the builders are playing Telephone with product management and customers.

Navigating little waterfalls

Focus your teams on laying out the course for an epic in the first sprint of dev work. A particular story may be unworkable or a design might need to be completely re-thought. Ideally that would've been found before the canoe was in the water, but the next best time is the present. You want to reduce the number of items that are reworked or added each day as quickly as possible, because hitting that fixed-scope is going to be what accelerates you to the finish line.

For the product manager, an MVP contains a minimum complete set of user-stories that improve a user's experience over their baseline. "Cutting scope" at the product level is an illusion based on misunderstanding the product definition vs its implementation. Engineers and designers will outline their approach and may ask for concessions that reduce the implementation time, but the actual scope of work remains the same.

Not every team is going to go through tickets at the same speed. Not every team is going to dial in the work to be done at the same speed or add engineering tasks at the same rate. Each major feature in your products is a different level of maturity and complexity, approaching a different problem for a different set of users.

They're going to have different profiles and that's fine. The goal with your measurements is to establish a baseline per team or per product, not to measure the delta between your team and an ideal or compare them to each other. If a team starts doing suddenly worse than it has before on fixing the scope or it starts adding engineering subtasks at a late stage in the game, you have a way to say "This isn't normal, what's wrong?" and improve.

For my money, the best unit of work to measure is the epic, but there are things that don't fit in epics. Drop-everything bugs come in. Vendors you integrate with change their products. Tooling and library dependencies age. Marketing updates branding. Sales is pushing into a new market and needs a new language pack. These tasks make up a sort of base-cost of operations and maintenance. You can categorize them differently and measure bug rates and so forth, but in the end what you want to know is how much of an average month or sprint or quarter is taken up O&M work vs. Developmental work.

I've divided this up various ways over the years, but I've found it really doesn't matter how you divide it up. Over-measuring doesn't clear up the signal. If there is an uptick in how much time is being spent on this by a team or on a particular product, dig in and figure out why and whether there's an opportunity to bring it down to or below baseline. It could be tech debt. It could be that a vendor has become unreliable. It could be a communications or employee engagement breakdown.

Once you have baselines you can make a prediction for new work. For our example above:

• The calendar team clears 15-25 tasks per week
• The number of implementation tasks will grow from the initial dev-ready set by an average of 60% in the first sprint, 20% in the second sprint, and then fall off.
• The calendar team spends 20% of its task clearance on O&M.

We just scoped out giving personal fitness coaches a way to schedule with their clients, and we have 32 tasks. That means that at the end of the first sprint we should have 51 total tasks and at the we should expect about 65-70 tasks in the epic by the time it's completed and shipped. That's about 6 weeks worth of work to get it to UAT using the lower numbers, and accounting for the O&M overhead.

You can use that in your estimates to others, or you can build in some wiggle room but keep in mind that projects usually take as long as you give them. You can even use statistical methods to build that wiggle room, and the more data you have, the better those estimates will be.

Recap

I know that's a lot, so I want to summarize the main points and give you a Start-Stop-Continue.

Start

Predicting completion based on past performance.

Organize work into epics. Epics are populated with user stories that describe a good user outcome. Then engineering and design work with the people defining the product (including customers where appropriate) to determine an approach and an initial set of development tasks that goes from breaking ground to shipped product. Once it's agreed upon, it goes into development and while the development tasks may change, the outcome should not.

Measure:

• Changes to the outcome (user stories or overall goal) made after development starts.
• The rate of development and design tasks added over time.
• The rate development tasks are cleared overall.
• The average number of "overhead" tasks within that.

Once you have a baseline it's easy to provide completion estimates to people outside the team. It's also easy to figure out if a project is on or off track, to proactively communicate that, including how far off track it is. And you have the information to dig in with the right people on the team if that happens.

Stop

Stop doing point-poker and all forms of "narrative guessing" in creating development estimates.

Stop letting development start on an idea that's not ready.

Stop writing user-stories that are actually development tasks.

Stop product management from "people pleasing" and pre-emptively negotiating against itself with the development team.

Continue

Continue to provide completion estimates, but better ones.