When I first sat down to write about agile governance, my mixed metaphor and oxymoron meter started redlining. As I researched to see how prominent this topic has been, I encountered phrases like “tail wagging dog” and “are you kidding me?” It seems that the Agile advocates and the governance advocates don’t see eye to eye, if they even talk to each other at all. While several authors have addressed this topic (notably Dr. Peter Merrick in a recent article in Information Age), most of the coverage is theoretical. My goal in this post is to outline some practical approaches to ensure that the requirements of large-project governance can be met by engineering teams that employ Agile software development methodologies.

Let’s start with some working definitions. I define governance as accountability and risk management of IT projects and resources as perceived by non-engineering personnel. Agile can be defined as the set of engineering methodologies that rely on incremental development cycles, flexible long-term planning, and rigid short term-planning. Though many will comment on these definitions, they will serve for our purposes here.

Looking first at governance, it should not surprise anyone that the CFO or CEO of a Fortune 500 company might be interested in exactly how their $100M annual IT budget is being spent and how this spending benefits the company. That’s their job as representatives of the business interests of the organization. Similarly, the business stakeholders at an independent software vendor, such as account representatives and product managers, need visibility into the activities in engineering in order to communicate to customers, investors and other external stakeholders. Put another way, governance of engineering assets is a requirement for business – without it, the engineering or IT organization is operating in an opaque fashion without accountability.

On the agile side, it should also not be a surprise that after decades of being blamed for late projects, over budget software releases, and failure to meet customer requirements, engineering organizations are fast embracing agile as a way to manage software development projects. Starting with the axiom that requirements are rarely fully specified, agile provides a scheduling and requirements management framework that enables teams to work successfully on under-specified requirements today, and adjust their plans as requirements change tomorrow, next week, or next month. Put another way, agile solves the requirements management problem that every business has – without this solution, the engineering or IT organization may be transparent (“See, here’s our project plan”), but is likely not satisfying business requirements.

At this point, I am reminded of the old joke: Do you want to make God laugh? Just tell Him your plans. Businesses need to plan their activities to ensure that resources are accounted for and are working to further business goals. Engineering teams need to work on requirements that don’t change, so that the work is focused and efficient. The question then becomes, how to reconcile the governance requirements that business stakeholders have with the just-in-time requirements management approach of agile? There are three practical things that organizations can do to ensure that agile benefits not only engineering, but the business as a whole:

1. Agree on high-level requirements. This is more than just ‘software or salsa’. It means that someone in the organization (a product manager at an ISV, for example) must take ownership for extracting and documenting requirements at the highest level. The great myth around agile is that there is no documentation – not true. There is no extraneous documentation. Everyone needs to agree that the organization needs to build an X that meets these A,B,C customer requirements. Agile purists who resist the creation of this document should be taken to task, as should product managers and other business stakeholders who want to meddle in engineering implementation details.

2. Plan for change at the business level. Given high-level requirements, the temptation for business stakeholders is to say “I’m done with my part”. That’s not agile, nor is it realistic. If your engineering organization is working with 30 day sprints, it’s the job of the business representatives to actively evaluate not only the requirements in the backlog, but also the output of each sprint, so that the next 30 day sprint is planned in accordance with any new information about the business requirements of the project.

3. Make governance a part of the agile process. This is the hardest part for engineering organizations to swallow. But the engineering or IT budget isn’t a gift from Santa – it is a hard number that your VP of Engineering or CIO owns and is accountable for. So it makes sense to add a step in the process at sprint review time (the post-evaluation that occurs at the end of each sprint) to formally document the project progress in a form that is useful for the rest of the business. Included in this report should be at a minimum what features were completed, what the cost of these features are, what the rollout plan is for these features, and what percent of the known requirements have been satisfied to date. In this way, the internals of the agile method are only minimally exposed to business stakeholders, who may not know nor care what a sprint is, but the results as they affect the business as a whole are transparent.

I’m sure that readers could come up with a few more practices, or critique my brief set. That’s great – what is needed on this topic is cooperation and frank discussion, so that businesses can move forward with more efficient engineering organizations, and with improved ways of communicating engineering activities to a broader business audience. Ultimately, both agile and governance are about making sure the business is working on activities that add value to customers. So it is in the interest of both agile practitioners in engineering, and business stakeholders across the organization to work together to blend agile and governance in a practical way that maintains the efficiencies of agile, and also advances and meets the overall needs of the business.

Interested in more on this topic? Attend a free live webinar with guest speaker, Jeffrey Hammond, senior analyst, Forrester Research on Thursday, December 4 at 1:00 PM EST:  The Business Case for Pragmatic ALM: Agility with Governance

This Webinar is now available as a recording here: Pragmatic ALM

Do you ever feel like things are out of control on your project, that you are adrift in a sea of conflicting priorities and requests? Do you suddenly find out at the last minute that you are the bottleneck and everybody is breathing down your neck asking you what is taking you so long to create the moveStuff() method but you had no idea that anybody even cared about moveStuff() or that you owned it? Do you ever find yourself in the exact opposite position, wondering why Sue and Bob didn’t get their stuff done that you need and then your boss walks by while you are surfing the net waiting for Sue and Bob? And who is Bob anyway?

The solution is simple! All you need to do is get everybody to move to Project. Well, if you have somebody you can spare full-time to keep Project up to date of course. Oh and I almost forgot, you’ll need to start using a requirements tool. But that’s it really, other than integrating them all together over the weekend and of course that’s assuming you’ve already gotten a CRM tool for workflow.

There is a simpler solution. It isn’t perfect, and it doesn’t solve all problems, but it can definitely provide the following benefits:

• reduce the chaos
• increase your vision into where you are and what’s going on
• reduce the number of status and/or project management meetings
• reduce the need to provide the same information over and over again
• simplify collaboration both locally and for distributed teams
• provide a more Agile workflow

The answer is to reduce the amount of rework that you are already doing. Right now you are probably storing defects in a defect tracking system, enhancements (aka RFEs, requirements, etc) in a requirements management tool (usually Excel or Word but sometimes an actual RM tool), and if you are using a project management tool it is probably MS-Project. All three of these product areas evolved to provide different aspects of project management for different groups of people and as a result they have lots of overlap. Considering how hard it is to coordinate three different systems, why not consider standardizing on one system for most of the work? The only question is, which system?

If we are going to try to do most of our project management work in a single tool, we should first decide what the interesting activities are. I believe they are: recording enhancement requests and defects as they are gathered by marketing or reported by users, load balancing, estimated completion calculation, critical path determination, work assignment, workflow, and reporting.

First let’s consider how well suited Project is for doing most or all of these tasks. Project is good at taking a small static collection of large tasks and doing load balancing, estimated completion, and critical path determination. Thus, it is mostly used for the very narrow task of project management of enhancements.

Next let’s consider requirements management. For whatever reason, most people use Excel or Word as their requirements management tool instead of a “real” requirements management tool. Excel and Word are just not appropriate for project management.

Lastly, there is defect tracking. A defect tracking system covers the assignment, tracking, workflow and reporting of defects. There is usually a higher volume of defects than enhancements, and they are usually smaller in scope and have a more complicated and often more time critical workflow. If it works well for defects, it should work equally well for enhancements.

Based on this analysis, it makes sense to extend the project management that you are already doing with a defect tracking system to include enhancements. A generic name for something that is either a requirement, enhancement, or defect is “work item.” By using work items to track all work, it is easy to see where you are and what remains to be done. Now you can use a similar workflow for enhancements as you do for defects, for instance from newly entered, to triaging, to assignment, to test development, to completion, to test, to integration, to delivery. You can easily run a query to see which work items have their code written but do not yet have any tests. Similarly, you can see which work items are done from a coding perspective and have tests but have not yet been verified as done by QA. This will give you a much more complete view of your overall project status and progress.

Whatever you are currently using for defect tracking it will be straightforward to start getting the benefits of managing defects and enhancements together. Just add a field that indicates if a work item is a defect or an enhancement. You may need to make a few more changes to accommodate a slightly different workflow for enhancements than you have for defects, but I think you’ll find it is worth the effort. For one example of how this can work, you can take a look at how AccuRev does it using AccuWorkflow.