The following post is an excerpt from a chapter we have authored in an upcoming book “The Encyclopedia of Software Engineering” to be published by Taylor and Francis.
Why Managing Agile Development Teams is Different
Agile methods are clearly different from their waterfall counterparts and operate from a different set of assumptions and design principles. In the example above, we saw no long requirements analysis phase, no requirements signoff, no phase gates, and no final handoff to QA. What we did see was a small cohesive team working in close cooperation; they did not have a real need for the heavy artifacts of the traditional process. This is not to say that Agile teams do not produce documentation. They can produce as many artifacts as is necessary and required. What is different is that Agile teams do not need to rely upon the heavy artifacts in order to move ahead. Traditional methods rely upon the heavy artifacts as the primary communication vehicle. Agile trades the necessity for heavy artifacts in exchange for frequent communication and quick feedback amongst a tight, well-formed team. In fact, in order for Agile processes to be successful, they typically require a continuous flow of software delivery, continuous feedback from stakeholders, short, time-boxed development cycles, continuous process improvement, and tight well-coordinated teams. These are the principles that allow the solution to quickly emerge from iterative development. The manager’s job then, when leading an Agile project, is to lead in a way that supports and encourages these principles. The trouble however is that most managers are not trained to embrace these principles and many managers are actually trained to do just the opposite: to keep some of these from ever happening in the first place. But in environments where these principles can flourish, we have seen great results from the utilization of agile methods.
Principles of Managing Agile Teams
1. Continuous Flow of Working Software
Continuous flow is the constant delivery of small, complete, functional units of code. The deliveries can be to a production environment, or more commonly, to a formal testing function. The goal in either case, is to deliver a steady stream of working code that can be used, tested, verified, or otherwise scrutinized for suitability. Compare this to huge batches of software that are delivered en masse into the testing phase all at once after months or years of development. By working in small batches, agile utilizes the science of queuing theory and inventory management to increase throughput and reduce cycle time. Small batches of software are faster to design, faster to develop, and faster to test. Although the small batch of requirements may not represent a fully functional system until later in the project, the ability to start getting feedback from users, testers, and environments is enormously valuable in getting an early assessment of the system’s suitability and fitness for purpose. Continuous flow is absolutely necessary in order to get feedback loops working in the iterative process. The manager’s job then is to plan and ensure delivery of working code every few weeks. This requires a different kind of project plan from what we see in traditional approaches.
2. Continuous Feedback
From continuous flow of delivery comes the capability for continuous feedback. And feedback is absolutely necessary for iterative methods to converge towards a solution. The ability to use or to test small incremental units of functionality provides enormous benefit in the ability to get quick and regular feedback from testers, users, architects, and others. Regular feedback from users and sponsors ensures that the team is understanding and implementing the requirements, not necessarily as they are stated on paper, but as they are actually needed by users. And early feedback from testers and architects can ensure that quality and design issues are addressed very early in the project instead of in the final phase just prior to deployment. In addition to these external feedback loops, the manager can also get continuous feedback from the team. Most agile processes including Scrum advocate regular process retrospectives in order to determine where process improvements can be made. By getting quick and regular feedback from both the external stakeholders and from the team, the manager can make sure that the team is always building the right product and is always building it in the best possible way.
3. Responding to Change
The market waits for no one. Technology changes, competition changes, organizations change, regulations change, and systems must change with them. The competitor that can regularly deliver effective change to the marketplace fastest often wins. Since Agile processes deliver in small batches of a few requirements at a time, there is more of an ability to adapt to changing requirements. When traditional teams put months of work into requirements and months into design, there is a tremendous amount of inertia to maintain the current path and to not invalidate the work that has been done to date. Huge investments have been made up front and strong change control is required to maintain the plan. However, Agile takes the attitude that the plan is not the goal; a suitable system is the goal and if requirements change or feedback from prior iterations uncovers new information, then the system and the team must adapt. And since agile teams do not typically invest huge amounts of time up front on detailed requirements, if a requirement does change, there is typically a minimal cost to allowing the change and satisfying the business need. The agile manager should not instinctively run towards tight change control when using the Agile model. Change is valuable and expected. The manager should follow-through and drive change that results from customer and team feedback. The discovery and delivery of change is evidence that the process is working. “But won’t customers continue to ask for change causing us to never deliver?” Theoretically, this is possible. But experience has shown that actually, the opposite happens. Customers ask for early delivery of the 20% of the requirements that deliver 80% of the system’s value. Using other methods, customers do not typically have the ability to exercise this option.
4. Solution Discovery
When continuous flow, continuous feedback, and the ability to respond to change are all combined, it is possible to use working software to incrementally discover the optimal solution. A team can quickly get working software into the hands of a tester or business partner and the partner can give quick feedback on the suitability of the solution. Changes can be continuously incorporated that allow the ideal solution to emerge. Traditional methods sometimes try to predict and plan the future in hopes that it will come true. “We think that this architecture will work, we think that this design will work, and we think that these are the right requirements.” Unfortunately, things do not always work out as planned. Agile’s ability to quickly implement a partial solution and change it, iterating towards better and better solutions, is in conflict with more traditional management thinking along the lines of “Get it right the first time”, “Eliminate Rework”, and “Plan the work and work the plan.” Just as a heat-seeking missile is initially pointed in the general direction of its target and then must follow the target as it moves, Agile methods seek to quickly deliver approximate solutions and then make adjustments in real time to hit changing customer needs. Managers of Agile projects will need to encourage the development of quick approximate initial solutions.
5. Continuous Improvement
Agile teams go through the complete software development process every few weeks. They perform some requirements analysis on a few requirements, and then quickly perform design, coding, and unit testing on those requirements. This continuous iteration across the full lifecycle provides many opportunities for process improvement. Instead of having one lessons-learned session at the end of a project, which is rarely reviewed in future projects, many agile teams perform lessons learned repeatedly throughout the project. This allows the teams to continuously improve their software development process while the project is still in progress. Changes can be made in mid-project to team communications, testing approaches, coding standards, build and deployment procedures, customer interaction, etc. The agile manager should make it a point to regularly inspect the process and adapt to the recommendations.
6. Real Teams
A real team is a group of dedicated cross-functional individuals that come together for close collaboration towards a common goal. A department of developers is not a team! But a group of developers, testers, and analysts all working side by side is a team. Teams work together as a unit. But the structure of many organizations makes it difficult to create true high performance project teams. The manager’s job in the agile environment is to try to craft an effective team from the organizational silos. Agile processes perform best when team members work in close proximity to one another for extended durations. The most agile teams work together in the same team room for hours a day. In this environment, there are rich, high fidelity conversations between customers, business analysts, developers, testers, and others. Now, perhaps for the first time ever, when a developer asks the business analyst or the customer a question about a requirement, the tester hears the same answer! In this environment, testers, developers, and analysts work very closely with one another to come to the same understanding of a requirement. This creates a richer understanding of the requirement and it eliminates a tremendous number of defects. These rich conversations lead to high performing teams that share a common understanding of the problem and the solution. The “us vs. them” mentality is often erased and an entrepreneurial spirit takes over. The team members come to know one another as individuals and this generates respect. Finally, Agile team members often have an option to expand their roles beyond the typical job description. In this team approach, business analysts can help to test, testers can help to elaborate on requirements, and developers get more involved in requirements and testing. Individuals take on more ownership, take on more varied duties, and in doing so, expand their skills and value. Compare this model to the handoff of heavy artifacts across organizational boundaries. Analysts work on a requirements document for months and then pass it on to the developers. They in turn work on the code for months and then pass it on to the testers, And in test, just before we go to release, we find out that nobody really understood the requirements in the same way. On Agile projects, the manager must ensure that close collaboration and rich communication are happening frequently amongst all core team members across a variety of disciplines.
Conclusion
Agile methods have a strong foundation in classical engineering theory and business theory resulting in a highly robust and effective process design. Agile thinking borrows from such varied disciplines as queuing theory, inventory management, control systems, finance, and organizational design to produce highly effective and responsive processes. The resulting processes however, use iterative methods that are derived from a different set of core principles than those of traditional methods. Given the different principles of Agile, different management techniques must be used in order to achieve success with Agile. Agile managers must organize the people and the work for quick and continuous delivery, for early and frequent feedback, for responsiveness to change, for solution discovery, for continuous process improvement, and must develop highly collaborative team models. With management’s support, these principles work harmoniously to produce high performing teams that frequently achieve substantial improvements in time-to-market, customer satisfaction, and team satisfaction.
