Agile Project Reporting and Metrics

“Agile processes are generative, not prescriptive. Processes need to evolve as needed, not be prescribed up-front. A prescriptive approach generates complex and complicated processes whereas a generative approach begins with a set of simple processes and adds others as they are needed.”

- Jim Highsmith, author of Agile Project Management

Agile recognizes that most effect software processes cannot be defined up-front but it is a continuous process. Traditional ALM process focuses on defining and enforcing processes and very little in identifying and delivering customers need.  Although they are aim to improve consistency and quality they often come at the expense of productivity and delivery. Traditional ALM has also used monolithic heavyweight tools, caused fragile builds and ineffective hand-offs between development, testing, deployment, and operations. Agile ALM frees companies from heavyweight processes and tools. Using continuous builds the customer is delivered regularly with working software which allow the customer to gauge the product which will be delivered and provide corrective measure beforehand.  While traditional ALM applied tools to enforce pre-defined standardized process, Agile ALM is the application of tools to support people and the processes that best suit the way they actually work when creating great software.  

A typical product development company has issue management process (Ticket Handling for Bugs/defects), Change/Feature Management process and new customer implementation process. In a product organization the PMO is a centralized resource for all the IT project teams to monitor project deliver with a wide range of projects with defined goals and timeframes. The PMO is responsible for (1) resource planning, including staff competencies, assignment, and capacity planning; (2) metrics and reporting to management (3) Definition and standardization project implementation and delivery processes, guidelines, training, and other project support services. Project managers need to  consistently communicate to and report on their projects across business units and development methodologies to the PMO so the PMO can present a single dashboard across  breadth of projects within a COO’s control, it is not surprising that the PMO is often heavy-handed, imposing a strict approach to metrics and guidance across all projects.

Among the above responsibilities the  greatest impact on Aglie project team is

1.       Providing dashboard to all stakeholders

2.       Providing metrics for process , project deliveries etc.

3.       Defining standard development practice which will support  and can be easily mapped to traditional project reporting and practice

Agile processes must participate within PMO infrastructure. This requires the Agile team willingness to conform to (or at least externally) traditional project management practices. Not only must the Agile project adapt the PMO must as well This will benefit from Agile projects’ ability to accurately measure the status of a project and adapt to changing requirements

Legacy PMO tracks project on the basis of project schedule, budget, issues and risk. This data is largely fed through project management tools (Ms Project etc). The PMO tracks time and cost estimate from project tasks dependencies among these task across the phase of the project life cycle.  Agile teams must communicate effectively, by using language and metrics that the PMO can appreciate. Agile team report on velocity (rate at which team delivers a story), burn charts(trend of remaining effort),test coverage and running test features. This do not effective help reporting in time, budget and risk analysis. Hence Agile team need to identify and bridge this reporting gap.

Sample  Agile Reporting

  

Release in Agile ALM mapped in Project Plan

Release Backlog 

Tradition Project Plan

 Mapping User Stories to Project Plan

Metrics such as earned value analysis (EVA) and cost to complete are certainly valuable to all stakeholders.  

Key Earned Value Management Terms Defined

(note: The term value in EVM does not refer to business value. The term refers to value as expressed by the project or program budget.)

Earned Value Term	Definition
Planned Value (PV)	The value of the work planned to be accomplished based on the budget (in dollars or hours)
Earned Value (EV)	The integrated value of work actually accomplished based on the budget (in dollars or hours)
Actual Cost (AC)	Actual Cost incurred for that increment of work
Budget At Complete (BAC)	The budget assigned to complete the work
Estimate To Complete (ETC)	The forecasted amount to complete remaining work (in dollars or hours) based on past performance.
Estimate At Complete (EAC)	The forecasted total amount for all work in the project plan based on past performance.

Earned Value Metrics

Metric	Formula	Metric Analysis
Planned Value	BAC * Planned Percent Complete	The planned value indicates how much value was planned to have been generated by a particular milestone or point in time.
Earned Value	BAC * Actual Percent Complete	The earned value indicates how much value has actually been generated at a particular milestone or point in time.
Cost Performance Index (CPI)	EV/AC	This metric indicates how many cents have been "earned" out of every dollar spent. It measures cost efficiencies.
Schedule Performance Index (SPI)	PV/AC	This metric measures schedule efficiency. It indicates how fast you are progressing against the rate of progress planned.
ETC	(BAC - EV)/CPI	This metric is the forecast amount to complete the remaining work.
EAC	BAC / CPI Or AC + ETC	Forecasted cost for the total planned work.

Agile story points can be used as the measure of work planned and work performed, providing the basis for all of the traditional earned value calculations. Percent complete can be derived by dividing the number of complete sprints by the total number of planned sprints. Actual percent complete can be derived by dividing the number of story points completed by the total number of story points planned for a release. To calculate the initial baseline in Agile 5 data points are required

1. The number of planned sprints in the release backlog
2. The length of each sprint in calendar days
3. The number of story points planned for the release backlog
4. The budget planned for the release backlog
5. Start date of the project

 To calculate EVM in Agile

1.    Number of sprints completed: This measures the expected percent complete when divided by the total number of sprints planned.

2.    Net Story points added (this can be a negative number if story points are removed from the release.) This reflects the net change in work planned for the release, in effect re baselining the release. Actual Cost to date At each boundary we are measuring results to date, therefore we use the cumulative actual cost as of that sprint boundary.

3.    Cumulative story points completed This measures the total amount of work completed for the release as of that sprint boundary.

Item	Definition
Budget At Complete (BAC)	The planned amount you expect to spend
Actual Cost (AC)	The actual cost to complete the work
PRSP	Planned Release Story Points for the release. Story points are defined at the Product Backlog level.
Expected Percent Complete (EPC)	Current Sprint(n) / Total planned Sprints
Actual Percent Complete (APC)	Story points completed / Total planned Story points
Planned Value (PV)	PV = BAC * EPC
Earned Value (EV)	EV = BAC * APC
Cost Performance Index (CPI)	CPI = EV / AC
Schedule Performance Index	CSPI = EV/PV

Agile Dashboard for tradition PMO

Team Dashboard using  TASK Board

In agile it is a common practice to maintain task board on walls to visualize the project status.  A visual process management system like dashboard (Task Board) which tells what to produce, when to produce it and how to produce

The project status as a dashboard.

Defect Metrics as per Release BackLog