No agile methodology quite enjoys the reputation and respect of eXtreme Programming (XP). Finding common ground with another XP nerd is a cheat code to trust and respect. “You did XP, so you must be cool.”
Some people are even calling for an XP revival as salvation from the agile apocalypse. “Revival” is a funny word choice. For non-XPers, the process might seem rather ceremonious and churchy with its many values, principles, rules, and practices, not to mention the zeal with which people evangelize its virtues.
If you used XP back in the day, it has probably informed your current practice. But what does it look like in the 21st century? What’s changed in the 25+ years since XP was invented? If we brought it back today, how might we modernize it?
My XP Origin Story
Since this is a post about biases and preferences, I should share mine and how they came about.
My first few hires gave me a pretty strong reality check. While I could produce code quickly, it wasn’t always the most maintainable or understandable. Much later, I’d label myself a legacy code firehose. We needed some standards so we could all work from the same understanding. Also, my managerial skills were nascent at best. I knew I wanted to create a proper team to complement and collaborate with each other. The vision was for the whole to be greater than the sum of its parts.
We needed a process, so I went shopping for one in the form of a book. Remember the bookselling chain Borders? The first book I purchased was “Agile Software Development with Scrum” by Ken Schwaber and Mike Beedle. I read that book in a few days. It gave me the concepts of iterative development and close communication, but I was left wanting more. Okay, fine, sprints, but what should we do inside these sprints?
Finding common ground with another XP nerd is a cheat code to trust and respect.
'You did XP, so you must be cool.'
I returned to Borders and found “eXtreme Programming Explained” by
(1st edition, act like you know). Jackpot! This was exactly what I was looking for. I bought multiple copies and sat with my teams for months as we applied and mastered the methodology's many practices together. Flash forward a couple of years, and we had almost ten thousand unit tests with only a few bugs reported every quarter.Since then, I’ve successfully applied these philosophies to many teams. You can’t tell me XP doesn’t work if/when a team commits. I’ll return to that last part toward the end.
Modernizing XP Practice
In terms of modernization, I’ll focus on XP’s twelve practices. Let’s review some of the more well-known ones and discuss how each has evolved.
Whole Team
The Whole Team concept in XP means that your team has all the perspectives and skills to produce valuable software. Engineering works closely with customers, product managers, designers, and other roles toward a common goal. This matched perfectly with my original team-building goal!
I still see forging Whole Teams as an evergreen practice, though I think the industry is backsliding a bit on this. Collaboration is a skill that takes work. People tend to avoid it. Compound this with the pandemic-era swing to remote only, and that skill gets harder to build.
The Whole Team concept works when individuals value a diversity of perspectives and share the belief that “we produce better results together.”
We practiced the Whole Team concept at VersionOne (now Digital.ai), an agile management/tracking tool SaaS company I worked at for six years. Like the well-known Pivotal Labs model, each team had a product manager, designer, tester, and several pairs of engineers. Each team collaborated frequently in discovery, pulling new stories, testing just before deployments, etc. While we’d go off to employ our special powers, we stayed in tight communication. The Whole Team concept works when individuals value a diversity of perspectives and share the belief that “we produce better results together.”
In my mind, Whole Team requires no modernization whatsoever. As senior software professionals, it’s our duty to invite (not force) people new to the industry to experiment with this approach. At Nerd/Noir, we fully embrace the concept. It’s in our DNA. If you’re not about it, there’s probably a better consulting partner for you.
Simple Design, Test-driven Development, Refactoring
Test-driven Development (TDD): you either love it or love to hate it. Write a failing test. Write just enough code to pass the test. Run all tests. Refactor as needed. Repeat.
The successes I’ve had applying TDD all look very similar. Many small tests describe your system's behavior. The greatest side effect of this is the executable documentation that serves as a change safety net. As such, I might start modernizing this by calling these tests “specifications.” Behavior-driven Development (BDD) and Specification by Example are second-wave practices that helped me understand and articulate the value of TDD.
The practices of simple design, TDD, and refactoring form a continuous loop that gives you emergent design. This design style acknowledges that you’ll have better information tomorrow than you have today, helping to keep your code pliant and receptive to that new knowledge. I like the humility and realism of the approach.
I’d modernize the design practice by augmenting it with other tools.
I’ve gotten a ton of mileage out of domain modeling as embodied by Domain-driven Design (DDD). DDD practitioners tend to spend more time understanding and analyzing the domain rather than the more bottom-up approach of TDD. Nothing prevents you from doing a bit of both.
I also still value the role of architecture. As such, I’d add tools such as the C4 Model and Event Storming that help a group visualize their systems and projects. I also want developers to understand the architectural styles, philosophies, and framework constraints they operate under, such as REST or API-first. I tend to go with the flow using community design recipes in front-end land, where frameworks suggest (enforce?) a design style that’s rarely worth fighting.
There are many new approaches to design and testing, so finding solutions that work in your context is very much a game of experimentation.
Design has grown in complexity as systems have become more sophisticated. Unlike when XP was created, we don’t develop within a single Small Talk image. We’re in the distributed-by-default era with many technical stacks. TDD and simple design are helpful, but we probably need a more comprehensive design strategy. That said, XP’s answers to design provide a tried-and-true selection of tactics.
Continuous Integration & Small Releases
What do you get when you combine continuous integration (CI) with small releases? Why continuous deployment (CD), of course! Continuous Deployment is table stakes these days.
Let’s talk about CI, though. CI has taken a hit since Git became the default choice for a version control system. Topic/feature branches have given way to the Pull/Merge Request lifestyle. Pull Requests create blocking problems as people wait for approvals or merges. This has yielded many new experiments and practices—async pull requests, ephemeral PR environments, and byzantine branching strategies.
I can’t help but wonder if we’re spiraling toward complexity here for no good reason.
I started with Git using trunk-based development, where we directly pushed to the main branch. This paired well with collaborative engineering practices such as ensemble or pair programming; the reviews are built into the programming process in real-time. As such, a separate code review handoff is an unnecessary waste.
If I were building a team today, I’d combine collaboration with trunk-based and have a source of truth (not eight) for integration. This isn’t a case of modernization; it’s one where modern tooling has introduced unnecessary complexity. It's time to get back to basics.
Pair Programming
While I was at VersionOne, we were a 100% pairing shop. It worked well on our teams of pretty senior developers. Declaring this preference during the interview process also repelled certain candidates. I’d call that filtering effect a win.
I’ve seen pairing be less successful when senior and junior people are put together. More experienced programmers tend to hog the keyboard and run with the problem while the junior person tunes out. While not always the case, this is a common failure mode. It’s especially apparent when your senior engineers value personal productivity over team development, teaching, and coaching.
The modern evolution is software teaming (ensemble/mob programming). Everyone working on the same problem on the same workstation simultaneously creating a state of psychological flow. It’s great for upskilling. It keeps WIP low, naturally. It solves many problems with onboarding. It deletes handoffs in favor of real-time design decision-making and checking. I friggin’ love it.
Software teaming suffers from the same cognitive leap as pair programming. Taylorist managers believe it halves their teams’ efficiency. Some senior soloists feel it dampens their individual flow. Newer engineers fear being found out as an “imposter.” The objections are mostly personal, and overcoming them takes a leap of faith or a great deal of digging into which faulty assumptions prevent people from experimenting. Either way, it’s a human problem.
For the record, I also think it’s fine for senior stacked teams to work solo using async ChatOps and PRs. Several friends who work in bigger tech-oriented firms report using this model effectively. These conditions are less common than people think, though. In every case, these teams are heavily stacked with senior engineers who trust each other to do the right thing. Oh, and they use pull/merge requests.
Sustainable Pace
The silver lining of the pandemic, we might call this work-life balance now. Building systems is a creative act. Getting results from this activity isn’t a matter of managing the iron triangle of scope, resources, and time. These days, I see fewer managers concerned about time utilization missed their true calling as law firm partners driving utilization.
In XP’s heyday, this wasn’t the case. Exploitation was pretty standard in the form of involuntary overtime and death marches. While this still occurs in pockets, the trend has shifted, and we owe a debt of gratitude to the agile community for helping to foment this change.
And The Other Stuff
There are even more practices. More than you can shake a stick at! In the interest of ending this post someday this month, I’ll go through them rapid-fire:
Collective Code Ownership, where everyone owns the code and we don’t create siloed knowledge, is a fantastic idea. We might look at InnerSourcing or Contract Testing in larger arrangements of teams developing in a micro-services architecture.
The Planning Game is an early XP style of developing a backlog. These days, I favor visual methods such as Storymapping or Event Storming.
Coding Standards - Use a linter, please. Embrace the idioms of your language and platform, please. You might even take a look at your SonarQube dashboard. I bet it’s lonely. Someone should check in on it occasionally.
System Metaphor was always the odd man out for me. I know XP co-innovator Ward Cunningham was big on metaphors. He came up with the technical debt metaphor. I never got much value from this practice, but your mileage may vary.
Want even more? Joshua Kerievsky created a variant of XP, Industrial XP, with 23 practices! But then again, it’s not the number of practices that counts; it’s how they work together to produce outcomes.
XP: Days of Future Present
Does XP need to be modernized or revived? In some cases, some slight tweaking is more than adequate. In others, getting back to its basics might be the right move. Either way, it still works as an approach to software development 25+ years down the line. People continue to laud the method, while processes like Scrum have been getting dunked on for as long as I can remember.
XP stands in stark contrast to Scrum for one clear reason—it provides sound advice on building software. It was built for a purpose and fits that niche particularly well.
In another wise move, the XP community wisely resisted modifying the process to scale.
At the 2019 Dojo Consortium Gathering, an intimate conference for immersive and experiential learning (dojo) practitioners, XP creator
shared the story of being offered a big bag of cash to scale XP by training a large company’s many teams. His proposal was to scale exponentially, as in the biological process of mitosis. Train a team, split the team (adding members new to XP), and repeat. Over time, the XP population grows in powers of two. Not surprisingly, this was deemed too slow by impatient leaders.Contrast this story with the false promise of scaling agility–big bang transformation, heavy process, training and certification, and small armies of coaches process compliance officers. Unrealistic scaling expectations are met with undeliverable promises.
XP wasn’t designed to scale coordination, command, nor control. It was designed for small, fast teams sharing the same values–Simplicity, Feedback, Communication, Courage, and Respect. Those values hold up as well today as they did decades ago. They hold up because they’re concerned with results, not sellability.
The Few, the Proud, the eXtreme Programmers
Setting aside stupid scaling discussions, what prevents a team with a high degree of autonomy from adopting or adapting XP today? What if you don’t have to sell your boss, only yourselves?
For me, XP was about rigor. If we’re going to do this, we’re going to do it right. I remember the early days with my teams trying to adopt the 12 practices as the meaning of XP’s values and principles started to rewrite our mindset. It wasn’t easy. But as with any conditioning, it got easier. I’ve heard it likened to a Marine Corps-like ethos. I’m not up on my Marines lore, but the analogy seems to jive.
Yes, you have to get over the basic training phase. Peer-to-peer and even sometimes outside support helps. But more than that? You have to get over yourself. You have to be willing to make mistakes in front of others. You have to be open to receiving wisdom from others. You have to be open to learning.
Whether or not XP experiences a second or third act, it happened and informed a solid core of people operating in the industry today. Let’s not luxuriate too much in nostalgia. Instead, put your energy behind the values, principles, and practices that yield results.