How can an organization reap the benefits of both traditional (waterfall) and more flexible (agile) project management techniques? The answer is through program management designed to deliver enhanced agility.
Agile project management practices are being widely adopted in the IT project management world. But classical agile methods were created for software development and are therefore not suitable for every situation and industry.
If classical Agile techniques do not work for every situation, how can organizations in diverse industries become more agile? The answer is by replicating the features of Agile software development techniques (the ones that contribute most to agility) and combining them with conventional project management practices.
Agile program management is structured to coordinate and direct projects with short planning windows, lighter documentation, and higher degrees of customer involvement. The result is a hybrid system of project delivery that is more agile than a traditional (waterfall) approach.
A common failing of long, waterfall-style projects is that these projects often develop "destination fixation." They emphasize the work that has been planned rather than building what is best for the organization. The waterfall approach encourages the team to follow the original plan (through destination fixation) rather than adapting to what the customer actually needs.
In waterfall, changes to the project are integrated into the project's plan through formal change controls. However, change controls are generally unsuccessful at managing substantive or continuous change because of the documentation involved and the effort required to obtain approvals.
The inertia in the waterfall approach is to "stay on plan," which prevents effective adaptation to changing realities. It is easier to follow the original, formally approved plan than it is to alter it. Enhanced agility breaks the cycle of focusing on long-term plans and allows the organization to focus on delivering benefits.
Implementing a full suite of classical Agile project management methodologies would require radical changes in the way projects are approved and managed in most organizations. As a result, many of the organizations that are attempting to "go Agile" are simply picking and choosing a few Agile-related techniques (such as daily scrums and the use of user stories) rather than converting to a full Agile methodology. The result is a blend of old and new techniques but not necessarily a more agile project management organization.
To be more agile is to be able to change direction, to react to reality, and to be responsive to discoveries. An agile project management organization is one that can pivot as needed in response to changes in technology and market forces.
Agility is the ability to rapidly change execution strategy. Trains running on railway tracks are not agile. Rabbits are. A project management organization is agile if the systems employed to authorize and manage projects are responsive to change.
An organization becomes more agile if the systems that support projects allow for more frequent re-evaluation and alteration of a project's execution strategy. Increased agility is not simply a consequence of adopting a few Agile project management tools. Agility is determined by how responsive or adaptive to changing conditions the overall system of project management is irrespective of the techniques employed.
To be more agile is within reach of any organization with or without the use of classical Agile software development techniques. Agility comes from having a responsive, flexible system of project management.
When considering whether to become more agile, the question an organization must answer is not whether they should adapt classical Agile software development methodologies but rather, what is the right level of agility to aspire to and how to achieve it? What is the best way to become suitably agile?
The keys to greater organizational agility are:
Shorter projects require less knowledge of the distant future. They are individually easier to plan and to execute successfully than projects with long planning windows.
Lighter documentation both speeds up the rate that projects can be planned and reduces the commitment to long-term planning that often prevents effective adaptation to changing situations.
Increased customer involvement helps to ensure that final outcomes deliver the maximum benefits to the intended audience. It reduces "destination fixation" in which a project builds whatever was originally conceived rather than what was the best solution at the time of completion.
Collectively, short projects, light documentation, and enhanced customer involvement can be combined to deliver a more agile project management environment than would be possible with the standard, long-term waterfall approach.
A project management environment that employs shorter projects, light documentation, and customer involvement requires strong leadership. This leadership is delivered through program management. The program manager becomes the connection between senior management and the project teams. They serve as the "voice of the customer," ensuring that project efforts remain focused on the best possible outcomes given current conditions.
In Agile program management, projects are planned and managed using familiar waterfall practices (plan the work, work the plan). The difference is that these waterfall-style projects are short in duration (two to six weeks), and between each short project, there is the opportunity to reassess the overall program's strategy. These points of reassessment are called "strategic inflection points." It is through these strategic inflection points that an organization's project management system becomes more agile.
Figure 1 below illustrates an example of the classic Agile project management life cycle. The salient points are:
And the cycle repeats itself.
Figure 1. Agile Software Development – Project Management Life Cycle
The characteristics of Agile project management methodologies that contribute most to an organization's agility are:
Short iterations. Iterations, in the agile world, are just short projects by another name. They are actually waterfallstyle projects in the sense that they begin with detailed planning and rigid expectations of scope (stories that will be developed), time (iteration length) and cost (resources available during the iteration). But, of course, they are very short, so the problems of rigid planning do not undermine the effective long-term flexibility of the overarching project.
The shorter the iterations, the more frequently the adaptation to change can take place. An agile project with iterations of five months each would be far less "agile" than a project with iterations of only two weeks. This point needs to be emphasized. The strategic inflection points between iterations (where teams can adapt the strategy to changes and re-plan) are what deliver agility.
The tools of agile software development (user roles, stories, rough estimates, etc.) are designed to support quick revisions of the project plan. A system of project controls is only agile if there are regular opportunities to revise the strategy before moving forward. Adaptation will only occur if the planning system lends itself to change.
The management of individual iterations is actually quite rigid. External stakeholders are not permitted to impose change on the project during an iteration. This is only allowed between iterations.
Change is restricted during iterations because change is disruptive to prepared plans (even for short projects). Changes discovered during an iteration are integrated into the overall plan through adaption at the next inflection point (between iterations).
Light documentation. If you are only planning in detail for one iteration at a time (two to six weeks), then it is not necessary to have a deep base of documentation.
Not having detailed plans long into the future supports adaptation and change. Change controls are brutal in conventional waterfall projects because of the complexity of acquiring approvals and of updating a myriad of documents. Reduce the length of your forecasts (plans) and it becomes easier for projects to move quickly and change strategy as necessary. Light documentation saves time at the outset by making it faster to get work started.
"Just in time" planning describes the Agile software development approach to project documentation. Long-term plans are high level with broadly defined objectives. They are strategic and focused on long-term outcomes. Short-term plans (iteration plans) are detailed; they are tactical and focused on near-term (two to six week) outcomes.
With just in time planning, scope is developed progressively. It is developed in detail as needed to support the next iteration and no more. Ambiguity in terms of what will happen in the distant future is allowed until the details are both necessary (one can't proceed without it) and available (stuff has been learned that allows for scope to be defined). Planning for an iteration (i.e., the next mini project) proceeds when enough is known to be sure that the iteration will add value to the overall project.
Customer focus. Agile project management methodologies depend on continuous customer involvement. This is achieved in three ways:
First, the customer (or their representative) participates in the planning of each iteration by helping set priorities and answering questions.
Second, the customer (or their representative) is expected to be available to the development team while work is underway.
Third, the project's product-to-date is reviewed with the customer at the end of each iteration. These reviews are structured to ensure that the team understands what went right and what went wrong. They serve to both initiate corrective behavior on the part of the team and to encourage customer involvement.
By keeping the customer continuously involved, agile teams demonstrate that they are working for the customer. Thus, the agile approach encourages a focus on benefits delivery. Continuous customer involvement is a key success factor for agile projects. It both reduces the need for advanced requirements documentation and it ensures that the customer's current perceptions are used to drive design.
In general, IT projects move quickly and suffer from a lot of complexity. Complexity is derived from such things as having numerous stakeholders, involving new technologies, dealing with overlapping systems, and requiring diverse skill set. Conflicts between development work and ongoing operations also contribute to complexity. Making matters worse is the fast pace of technical innovation and ever-evolving stakeholder expectations.
High complexity makes it difficult to define a project's scope of work in advance and, therefore, to estimate the cost and effort required to complete that work; subsequently, this is a big problem because these are necessary ingredients of the traditional waterfall approach to project management.
By contrast, Agile project management techniques allow requirements to be refined progressively rather than upfront. The execution strategy (project plan) is permitted to evolve as knowledge about the scope evolves. This progressive evolution of scope allows work to begin quickly when just enough is known to get started and without knowing everything. It supports "just in time" planning.
However, there is a reason why Agile methodologies have not been universally embraced. Agile projects do not begin with a detailed plan; they do not begin with a promise of what will be accomplished in the end like a waterfall system of project management does, and that is a huge hurdle for the people approving projects.
The waterfall approach to project planning creates a detailed forecast. This gives the people paying for projects a greater sense of confidence in what will be accomplished and of the total cost to which they are committing. There is the perception that a waterfall-style approach to project planning reduces uncertainty and, thus, reduces the risk of failure. Unfortunately, history tells us that it is not true. Waterfall planning does not reduce the risk of failure in a rapidly changing, complex environment.
Waterfall project management is based on heavy up-front analysis, which leads to detailed "plans." A project's plan is a forecast of future outcomes in terms of the work that must be completed, the cost involved, and the schedule for all that work. The waterfall approach is to plan (or attempt to predict) exactly what will be done and then execute the project according to the plan. Detailed predictions offer a conceptual logic that provides comfort (i.e., a perception of risk reduction) to senior managers who are funding the projects.
Unfortunately, the waterfall approach is based on the presumption that scope can be unambiguously defined at the beginning of a project. The waterfall planning system depends on being able to define all of the work (or scope) in advance.
And that is the problem.
IT projects are rarely given the time necessary for sufficient analysis to produce accurate forecasts. Even if time were available, changing technology and operational complexity inherent in many IT projects makes detailed scope definition, in advance of starting the work, virtually impossible. By the time an exhaustive study of requirements is complete, the requirements are already out of date because of technical and market changes. Painstaking requirements analysis has not proven to be a guarantee of IT project success – just delays, frustration, and misdirected efforts.
Waterfall project management requires settings that are stable enough to support a methodical approach to planning. Suitable stability is not the norm in the rapidly changing IT sector. Therefore, what is needed is an updated waterfall model.
Enter Agile program management.
Can we have the familiarity of a waterfall approach and some of the agility promised by classical Agile methodologies? Yes, by using numerous short projects (rather than big long ones) that are strategically coordinated as part of an overarching agile program.
Figure 2 below illustrates an example of an agile program management life cycle.
Figure 2. Agile Program Management Life Cycle
Individual projects are short in duration but are planned in the conventional waterfall way. Between projects, there is the opportunity to adapt to revelations both from individual projects and based on the combined outcomes of multiple projects.
Light documentation is made possible by shorter planning windows and careful stewardship by the program managers.
Customer involvement through the program manager as the 'voice of the customer' ensures that the team never loses sight of the overall objectives.
Agile program management delivers the crucial elements that underlie the agility of Agile software development methods without having to adopt all of the agile tools and techniques.
To guide and direct. Careful oversight is key to the success of agile programs. Project teams are able to focus on their small world (only their projects) because the bigger picture, and stakeholders are generally managed by the program team.
The program manager serves as both mentor and customer. They become the key stakeholder of the short projects, act as the "voice of the customer," and keep the project's focus on short-term objectives while simultaneously keeping the program's emphasis on long-term benefits.
To monitor and control. The program manager monitors project efforts and outcomes to make sure each small project contributes incremental benefits. They control re-planning and strategy development between projects to ensure delivery of program benefits.
Success with the Agile program management approach hinges on:
In the waterfall world, risks are managed by having a detailed plan and "fixed priced" relationships between the project team and sponsoring organization. A waterfall project plan binds the team to a specific set of outcomes (scope, time, and cost). The stakeholders, therefore, think the risk of failure has been reduced. But history tells us otherwise. You cannot plan for what you do not know, and you cannot find out without plenty of time and resources.
Faced with operational complexity and rapidly changing environments, organizations need to manage risk differently. How successful would army generals be if they planned every troop movement in advance of a battle, and if, during battle, any changes to the battle plan had to be approved through a laborious military hierarchy? Not very.
Success in a fluid environment comes from planning only as far into the future as there is certainty. Beyond that, you need to be flexible, and that flexibility comes at the cost of not being able to demand long term, rigid plans from project teams.
Greater organizational agility reduces risk, but it requires a substantive change in attitudes towards risk management, which also requires a new approach to how projects and programs are funded and controlled.
Agility means being able to adapt to changing conditions and being responsive to evolving needs. In a rapidly changing, complex environment, greater agility contributes to lower project costs and more effective project outcomes. Greater agility delivers this by making project management systems more responsive to discoveries, revelations, and changes that arise during execution.
Every organization can be more agile but only to the extent that they are willing to plan tactically in the short run and strategically in the long run. The agile approach requires near-term specifics and long-term generalizations.
What holds organizations back from being more agile is funding models that require waterfall style project planning complete with heavy up-front documentation that produces rigid plans for outcomes long into the future. These detailed execution strategies create momentum for investing in solutions that quickly go out of date but cannot be derailed. This results in "destination fixation" and projects that build the wrong things.
The "magic bullet" is Agile program management. Agile program management enables the delivery of benefits through a series of shorter projects that are strategically tied together. These short projects allow the overall execution strategy to be regularly revised. Documentation is kept light to support flexible planning. Detailed documentation (such as plans and expectations) is only developed for near-term deliverables, and general documentation is only created for long-term outcomes. The combination is a more agile project delivery system that better serves ever-evolving customer needs emphasizing the short-term without losing sight of the long-term.
The agile program mantra: Near term specifics, long term objectives.
Brian has been a contract instructor for Global Knowledge since 1999. He divides his time between management consulting, project management, technical writing, and professional development training.
Brian is a "serial entrepreneur." He has started companies in such diverse fields as fish farming, woodwork, gift manufacturing, and catering. He is the author of numerous training courses relating to professional skills, project management, and decision making.