Comprehensive software reviews to make better IT decisions
Agifall: The Portfolio Strikes Back
The Agile method is built on an admirable and useful set of values, and the software developer experience makes me miss most of the 1980s. No, not the Wham! and Tears For Fears parts. Eeww. I’m talking about the part where developers are dedicated to the project. One task at a time, one project at a time. One boss. If we set aside Neil Young’s forays into techno and 50s music, I was much less confused in the 80s than I am now.
Don’t get me wrong – the strict adherence to the project management lifecycle was awkward. I worked for a big bank and once spent an entire year in design without writing much (if any) code. The customer/product owner was livid: not at me, I trust, but at the process. He had to go about four years without product enhancements while we rigorously built a new system. We built the heck out of it. Killed it. After all, we had four years. And we didn’t call it “Waterfall” yet. It was simply the way that one ran a software development project.
This new world of Agile really does borrow from the great parts of the pre-Internet era by allowing people to focus on fewer tasks, at least conceptually. It is supposed to reduce the wasted time and leave people alone to produce value.
Sadly, that “leave people alone” pendulum has swung too far. The executive decision maker still wants to know what you’re doing, when it’s going to be done, and what it’s going to cost. For the Agile purist, the answers are, respectively: whatever we decide in sprint planning; whenever you tell us to stop, and; whatever it has costed when we stop. This is not making Agile a popular conversation topic among affected executives.
In that respect, we can see a bit of a divide in the success of Agile in the financial services industry. The very large organizations have clung to enough oversight to retain control. Smaller firms are further down the road with Agile, but many have hit the ditch.
Most Agile is like Newtonian Waterfall
The vast majority of organizations have ventured into the world of Agile. Fully Agile shops aren’t all that common, and we tend to say that our clients have mostly mixed methodology project portfolios.
But it’s also very uncommon to find IT shops that would call themselves “purely” Agile. Most say they’re “Agile-like” or “Agile lite” or “Agile-ish” or “progressing toward Agile.”
So here we are, with an incredibly light set of principles to follow. Well, they’re perhaps better described as values. And really, how hard can it be to claim that you value individuals and interactions over processes and tools, working software over comprehensive documentation, customer collaboration over contract negotiation, and responding to change over following a plan? Yet, most practitioners nervously shrug off the label and say, “we’re not really pure Agile.” Perhaps they’ve been called out by an Agile purist for not doing it right.
How? What is “not doing it right”? Aside from the problems of unstable teams being assigned too many projects and too much non-project work, there’s a more fundamental problem: Agile still wants to be deterministic. It’s a useful way to organize the efforts of competent professionals, and it expects outcomes that are not random.
The definition of Agile is so indirect and lacking in prescriptiveness that you get your hand slapped for calling it a methodology. “Framework” is OK, but one gets the sense that you’re better off calling it a “set of notions, or directional principles.”
Agile wants a carefully constructed list of stuff that’s going to get done in a fixed amount of time. Sprint planning. When we talk to people who shrug off the Agile label, they’re most likely to have these three conditions:
- Sprint planning doesn’t get done, or fails to yield a realistic set of expectations for the upcoming sprint. There’s probably a decent list to work from, but little is done to size the demand to the supply (i.e. human effort hours). So, it’s more like timeboxing without knowing the size of the box or what’s in it.
- People are assigned to multiple projects and left alone to balance their work. At the risk of beating the “resource capacity” horse to death, you can’t have deterministic work management without sizing the supply.
- People are still expected to do support work as needed. That work is almost always more urgent than their project work, whether the project is Agile or Waterfall or something else. The fundamentals of modern knowledge work haven’t changed: project work is what you do when you have nothing else to do.
From where I sit, there appears to be a common (though not universal) barrier to success for Agile projects: product ownership fails to delineate the sprints. The majority of our candid conversations with practitioners and PMOs indicate that the product ownership role is poorly defined and poorly executed.
Consequently, many sprints happen without a crisp ending to the prior sprint. There’s no clear acknowledgement of what was completed and no deliberate crafting of priorities for the upcoming sprint. And while it makes sense to call this out, the project team has little motivation to blow the whistle. It’s like expecting the dog to self-report when its owner doesn’t use a leash. Woof.
I describe the application of these principles as “Newtonian Waterfall.” In other words, the project is cascading through progressive phases like a Waterfall project, but with no control points between the levels. All stage, no gate, no checkpoints, just free-flowing gravitational effects taking hold of progressively completed bits of work.
When you got the “self-managing teams” message, you got the “self” part right and forgot the “managing” part. Who’s controlling the resources and expenditures? Surely nobody who’s being held accountable… and surely nobody who was supposed to have the authority.
Waterfall was supposed to be more of a canal system
Waterfall was supposed to be reasonably controlled, in truth more like a lock system than a tall cliff. Gravity wasn’t supposed to bring you through the process: successfully passing a checkpoint was supposed to be that trigger.
The system was a beast and difficult to balance. In many projects, the delineation between phases of the SDLC was violated in both directions. It was common to be in the internal design phase while some of the functional design was incomplete and some pieces were being coded.
Shhhhh! Don’t tell anyone.
Let’s document this before too much time passes and the actual history gets lost: that old paradigm was largely disrespected and frequently ignored. We didn’t speak too openly about it, but we slithered out of the shackles of that old project lifecycle quite often.
The theory dictated that business requirements be stable going into functional design. And the functional design was to be stable before the internal design, etc. The theory was hard to demonstrate while scaling function and team size, and the concept of stable requirements was utterly laughable.
No, the smart (or perhaps, practical) way to drive your project was to mitigate risks by bouncing forward and backward as needed. A little prototyping here, some secret integration testing there, and definitely a bit of advanced pre-acceptance testing. And for sure, quietly and respectfully accommodating a change to the requirements without chipping off even more of the customer’s fragile mojo by dragging them in front of a military tribunal and demanding penance for their contemptuous inability to design software and commit to it forever. And ever.
On a project of any size, success came from knowing how and when to mitigate and eliminate risks by blending the phases. If it broke the idealized notion of sequence, so be it.
A blend is taking over: Agifall?
Increasingly, people are blending these organizing concepts to find something that works. Most commonly, people say “We’re Waterfall until design is finalized, then Agile from there.” And yes, they often call it “wagile” because it starts with the Waterfall part.
I’m pushing back a bit by calling it Agifall because it’s still a mashup of phases, in practice. Looking over the shoulder of these new-age projects, it sure looks like there’s some iterative development happening during those early design phases. And it sure looks like the design need not be carved in stone if we discover better ideas later in the project.
Regardless of labels, the process that works needs to accommodate phase blending – forward to mitigate risk and backward to accommodate evolving insight. If the process is visible and monitored, we should be able to handle the malleability.
And you really do need some controls and oversight. How did people who aren’t authorized to buy a new mouse for their ThinkPad (or install the mouse drivers on that ThinkPad) come to direct millions of dollars of resource time in these Agile projects without any controls?
Best get that part nailed down a bit before the shareholders figure it out.
Whose idea was it to abandon all commitment to scope, time, and cost? Newsflash: it wasn’t the executive decision maker. The pendulum is swinging back from the Agile dream to the executive reality. More and more leadership teams are asking for a Waterfall-ish commitment up the hierarchy to the portfolio with little concern about the project management style. To quote my favorite boss, “I don’t care if you row your project up to the dock in a birchbark canoe. But it’s due on the 15th.”
Recommendations
Avoid a dogmatic adherence to the organizing concepts:
- Get, share, monitor, and evolve the team’s commitment to scope, budget, and timelines to satisfy the duties of management.
- Mitigate risk early by iteratively prototyping, integrating, testing, etc.
- Use traditional change control (where it makes sense) to avoid the chaos of “we’re Agile – we don’t do planning and design.”
Bottom Line
As a portfolio manager, your job is to help the decision makers do their job. That means setting expectations that are commensurate with the resourcing being committed. The executives are wise to the game, and they’re losing their appetite to approve years of funding with no commitment in return.
Want to Know More?
Balance Supply and Demand with Realistic Resource Management Practices