What would it take to build a product development system with high agility?
I’ll start with my definition of agility:
Agility measures a group’s ability (team, organization, etc.) to implement new learnings with minimal disruption at a reasonable cost.
Agility is a property of a development system. Some systems are difficult to change (low agility), while others are easy to change (high agility). What prompts change? Learning.
Learning takes many forms, but common examples include changes in stakeholder priorities or market conditions, insights into actual versus imagined customer needs and behaviors, and a deeper understanding of the technical ecosystem in which your digital goods and services live.
High-agility groups implement learning faster with less churn and drama, which can be a competitive advantage. Their “turn on a dime” capability serves them well in domains where we regularly uncover new insights, problems, and opportunities.
Before moving on, I should clarify that I’m using the word system in a broad context. A product development system comprises many sub-systems, including but not limited to people (teams, leaders, customers), artifacts (code, docs, etc.), and workflow. Today’s rant is mainly about a system's people and workflow.
The Great Agile™ Backlash of 2024 2010
I thought my definition of agility was pretty desirable. Still do. Why wouldn’t almost every company seek to transmute change from a disruptive and dangerous nuisance to a source of innovation and growth?
It’s not for lack of trying! Enterprises have been throwing money at this quest for years. Some companies succeeded, moving beyond the product of agile and having achieved a cultural property of agility. Some are still trying. However, many firms threw their hands up and returned to their old bureaucratic or pathological ways. I’d argue that the failures over-indexed on the agile product and the process, not the property of agility.
It’s 2024, and big companies are jettisoning their investment in agile. They’re deleting roles like coach and scrum master. Executives are mandating that their managers must provide coaching and team organization services and get involved in developing their people. Imagine that. Also, if expenses have just been cut and not reallocated to developing these new manager-coaches, good luck with that.
Here’s the thing—this backlash has been boiling for well over a decade. If you’re new here, welcome to the party!
I remember complaining about capital-A Agile years ago, as the ubiquity of Scrum eclipsed the valuable parts of eXtreme Programming. Soon after, the macho men of software craftsmanship took their turn complaining about agile’s lack of technical practices. That’s about when I tuned out, and I’m sure some people can out-hipster me on this claim.
Agile went establishment. It became the new dictator to usurp for a few good reasons and a few suspect ones – taking a shot at the title is always a good startup idea for a personal brand if your conscience will allow you to be a troll.
Agile turned into a product, something for sale. Like any product, it started to suck after a critical mass of opportunistic jokers and clowns got involved. There’s a little bit of Lloyd Dobler from “Say Anything” going on —
“I don't want to sell anything, buy anything, or process anything as a career. I don't want to sell anything bought or processed, or buy anything sold or processed, or process anything sold, bought, or processed, or repair anything sold, bought, or processed.”
But big dumb companies and their strong capacity to groupthink played a big role, too. Agile became a cargo cult (Spotify Model, y’all!) and certification (SAFe 8.0 is out!) mill. Enterprise consumers bent it into the shape of its intended replacement: old-school program management.
Like all revolutions, it became lame, laughable, and sometimes traumatic as it scaled, replacing the language, not the substance, of the status quo. Project management isn’t back, baby! Turns out it never went away.
Too much has already been said on this topic. I’m bored of hearing about it, much less writing about it! It’s a tired meme. We should all move on.
Revolutionary Tactics
Speaking of revolutions, good revolutionaries employ a tried-and-true playbook of dirty tricks. A common tactic involves establishing a common enemy to rally your troops around: a scapegoat or boogeyman to concentrate the fire of your budding group psychology’s malice or fear.
In the first decade or so of the agile software development revolution, the waterfall model (sequential phases of development) became the go-to common enemy. Many mediocre and repetitive conference presentations sought new, interesting ways to prove that “waterfall is bad.” Resorting to LEGO® to explain agility’s benefits by way of analogy was, for me, the zenith of these stupid moments.
But why, exactly, is waterfall bad? It tends to bake in the assumptions made early in a project’s lifecycle. Requirements become fixed, and construction proceeds without the benefit of new information or feedback. The biggest risk in employing this workflow is assuming you have it all figured out. Some would label this risk overconfidence. I prefer arrogance.
The keyword here is “risk.” If you can (a) know all risks ahead of time, (b) sense and respond to emerging risks as they ebb and flow, and (c) develop an avoidance or mitigation strategy for each of these risks, if you can do all that, maybe waterfall is fine. It’s a process optimized for different properties, however rare.
Anyway, back to the common enemy tactic. Establishing a common enemy helps focus your forces, but there are tradeoffs, as with any tactic.
Common enemy trades simplistic messaging for critical thinking. Several generations later, all agile newcomers were onboarded with the same mantra (“waterfall is bad”). Well, why should we expect noob agilists to answer the questions of when or why waterfall development is the wrong choice?
Do you use a common enemy in your revolutionary activities? Maybe something to consider the next time you write that LinkedIn shitpost intended to rally your base.
Choices & Balancing Acts
Do you have a straightforward problem? Is it clear with pockets of complexity? What properties would you try to optimize your system for in this case? Probably predictability of delivery date and cost. Project management is fine, and waterfall sometimes even works, so long as you keep those waterfalls small just to be sure a FUBAR (FOOBAR?) moment doesn’t suddenly appear in month 14 of your project (aka day one of your death march).
For the rest of us, those with a healthy skepticism of anyone’s absolute certainty (including our own), we might do what we can to ensure our workflow is optimized for agility.
Of course, we’ll have to balance several properties over time. Security, resiliency, usability, scalability, cost, and others are all important. We’re back to critical thinking. Can you weigh the benefits of agility without the trappings of the failed agile-industrial complex? I bet you can. I have faith in you. I want to believe.
Agility from First Principles
Let’s do a little thinking together. Imagine you’re developing a new distributed system, using many new tools and a new platform. You want to replace a mysterious legacy system. It’s poorly documented, a rat’s nest of code, and its sages have long since retired. Sounds like a lot of learning. Fun!
Remember my definition from earlier?
Agility measures a group’s ability (team, organization, etc.) to implement new learnings with minimal disruption at a reasonable cost. – Me, 2024
What will we need if this is a property we want to optimize for? A few things spring to mind:
Get frequent feedback on what we’re working on from stakeholders, especially users and customers.
Keep commitments small to invite more frequent feedback, make mistakes and misunderstandings small, and keep design choices reversible.
Focus on validating our product assumptions and technical choices, especially early in development.
We have the beginnings of some principles that will guide our choices. How, then, do we set up our system? Things like time boxes (iterations) and stakeholder involvement (testing, sharing sessions, user interviews) would help. Technically, continuous deployment and domain-driven design might be good things to check out.
Pay attention to the mechanics and the analysis I’m using. Our understanding of our context drives us to design principles, and only then do we go shopping for practices and tools—all because we chose to optimize for change. In this scenario, the change comes in the form of “lots to learn here.”
Optimizing for agility is often optimizing for uncertainty. And, you’re probably operating in a domain of uncertainty, whether you want to acknowledge it or not. If your personal baggage makes the “a-word” too cringy, try on “learning” or “uncertainty” for size.