The Agile Balanced Scorecard

The U.S. Federal Reserve Board keeps close tabs on two high-level measures of economic performance: GDP and inflation. While growing GDP and simultaneously keeping inflation in check are the end goals, the FRB looks at many other measures as well. That is because GDP and inflation are “lagging indicators” of economic performance – they confirm longer-term trends but cannot predict them. For example, it took macro economists a full year to figure out that we were already in the current recession. So it would be foolish to focus on only these numbers because by the time they indicate that a change is required, it is already too late.

Enter “the leading indicators.” Leading indicators are measures that often give early signals as to where the economy is heading. By looking at such measures as housing starts, durable goods orders, warehouse inventories, unemployment rates, etc., the FRB can see early trends in where GDP and inflation may be headed and decide what actions to take.

What does all of this have to do with Agile and software development? Managers in corporate settings also need to think about leading and lagging indicators of software development performance. They are expected to develop some foresight into what is to come and take proactive measures as appropriate. What metrics can act as early warning indicators of future software development performance – and which can tell us whether or not we were ultimately successful? We will discuss some potential leading and lagging indicators that may be used to assess software development success, particularly in the Agile development space.

Yet another way to think about software development metrics is to consider the sometimes-opposing forces of effectiveness and efficiency. Efficiency measures are focused on how well we planned and used our resources to deliver our product. Budgets, schedules, labor hours, average time per software release, average spend per software release, SLOC, etc., are common examples of software efficiency metrics – here are many more. Effectiveness measures, on the other hand, indicate whether or not our IT solutions meet our business needs. Examples include customer acquisition and retention rates, customer satisfaction, revenue or cost savings, and profitability.

In developing a measurement system for IT projects, we need to think about how to assess the following two questions: “Did we build the right thing?” – our effectiveness – and “Did we build it the right way?” – our efficiency. If you build the right product at the right time, you will have a high likelihood of success and can then work to improve your efficiencies. But if you build a product that few customers want, you can be highly efficient and still suffer a massive loss.

IT has a history of putting more emphasis on efficiency metrics than on effectiveness metrics – but I believe that effectiveness trumps efficiency almost every time.

Robert Kaplan and David Norton introduced the Balanced Scorecard approach to organizational strategy design and measurement in 1992. As the name suggests, the idea is that organizations need to balance their strategies and their measurement systems across a spectrum of parameters to achieve long-term success. Since then, the framework has grown into a strategic management system that can be scaled up and down the organization and also used for holistic organizational measurement across the following four basic categories.

Financial – Measures of financial performance. At the corporate level, these might be revenues, profit, earnings per share, etc.

Customer – Measures of customer satisfaction including customer growth, retention, satisfaction, equity, and others.

Innovation and Learning – Measures of internal skills and capabilities. Examples here might be the percentage of revenue from new products, level of education in our workforce, number of training hours delivered to employees, etc.

Internal Business Processes – Measures of processes usually in terms of efficiency such as time-to-market, inventory, throughput, defect rates, etc.

The Balanced Scorecard was designed to be scalable across the organization. That is, high-level corporate measures along these four categories could be pushed down to the business-unit level and functional levels, and also rolled up and integrated organizationwide. In this system, individual departments could contribute to financial performance, customer satisfaction, internal business process improvement and innovation and learning. These four categories are just suggestions, and organizations should modify them to fit their particular needs.

An Agile Balanced Scorecard

All three of these frameworks – leading and lagging indicators, efficiency and effectiveness, and the Balanced Scorecard – have merit. But is it possible to achieve a holistic view across all of these important measurements and make it work at the program and project level where most of the work actually gets done?

The Agile Balanced Scorecard takes the best of these approaches to form a unified measurement system at the project and program level. This is done by modifying its traditional four categories to better align with the way we typically think about projects. Our categories are:

Product: Are we building high-value products that are in demand by our customers?

Financial: Are we achieving planned-for financial results?

Team:Are we working well together as a team and addressing the needs of the various stakeholders?

Schedule:Are we on track to deliver within the schedule commitments?

You could stick with the original four categories of the scorecard, use these modified categories, or define your own when setting up your own scorecard.