We recently produced a webinar here at AccuRev, co-sponsored by Rally Software, about the Top 10 Factors for a Successful Agile Implementation.  Number 7 addresses the question of a tools stack. Once Agile coaching and training is complete, now we can think about tools.

Agile manifest declares:  ”Individuals and interactions over processes and tools.”

But we believe that this statement doesn’t mean you should forget your tools stack entirely. It’s really about making sure that your toolstack helps facilitate those interactions between individuals.

So it’s no surprise to us that when we hear about people rolling out Agile, they don’t even consider upgrading any of their tools. Studies have shown that new teams that do not invest in Agile tool sets, automated testing and continuous integration, start to struggle around the 10th iteration (4-5 months). The backlog of bugs and to-dos become so great that teams have to stop what they are doing and do a “maintenance” iteration, where they do massive amounts of QA or stop and do code reviews for an entire iterations. If we think about it, this kind of seems like traditional development model doesn’t it? Are you really Agile if you can’t deliver something which is “done” after every iteration?

The problem? Your current toolstack was designed by people who had traditional development models in mind.

Your tools don’t actually facilitate shorter development cycles, they actually hinder you by getting in your way. Your tool stack shouldn’t be the impediment in your Agile rollout.

Lets start with an Agile project management tool. This is the obvious tool to replace in your stack. Converting your traditional issue tracking system to an Agile one will give you the speed and visibility to help your teams practice Agile on a day to day basis.

Traditional source control management systems also hinder the process by following a “waterfall” like model of isolating code changes and leaving them on branches. Most SCM vendors know this is a problem, which is why their solution for an Agile branching methodology is to consolidate branches, or have entire teams work on one branch. This doesn’t make sense. All of your scrum teams should be able to function together and in parallel, while integrating work together on a regular basis.

Smaller iterations mean a faster paced development cycle. Some of the tasks that you used to do a handful of times over the course of a waterfall will now be done on a weekly, sometimes daily basis. This is where continuous integration and test orchestration come into play.

While you’re moving through a 2 week iteration, using continuous integration and automated tests suddenly becomes extremely important. Having these processes in place can help you avoid the 10th iteration problem we talked about earlier. You’re not going to have 4 months to regression test the application by hand in Agile.

The idea here is that your Agile tools must help you build a testable, deploy-able app every 2 weeks like clockwork. You will repeat this process over and over. Isn’t it time your toolstack provided you with clockwork development cycles?

by Rob Mohr

Do you have the “Admin Trigger” installed and running in your AccuRev environment?  I hope so!

The “Admin Trigger” is the best way for you to restrict those non-ACE’d (AccuRev Certified Engineer) users from wreaking havoc on the process you’ve meticulously designed and implemented for your organization.  Make sure you lock it down!  It’s really simple to do!

Now, I’m not talking about taking flexibility away from your developers, they’ll need to control certain aspects of the process too.  It’s up to you where the line is drawn in the stream hierarchy and the Admin Trigger is the chalk.

The equator is commonly established on the integration stream.  Since the globe in this case is AccuRev, the line of demarcation runs north and south.  To the West (upstream from integration), ACE’rs set the rules on workflow, code promotion, stream configurations, and general access control.  To the East (downstream from integration), developers and development teams are free to make their own decisions to best support their process, products, projects, components, patches, bug fixing and development activities.

Have you read the private prototyping or stream-per-task blogs by Dave Thomas?  These are good examples of how dev teams and developers control how their activities are organized using streams while adhering to the overall enterprise software development life cycle (SDLC).

The Admin Trigger is a simple if-then-else perl script that fires on the server whenever certain commands are  executed.  Out-of-the-box, the script restricts “admin type” commands such as creating users, groups, depots, etc without needing additional customization.

 @cmdlist = qw/mkuser chref chdepot chslice lsacl addmember
                  rmmember mkgroup mkdepot mktrig rmtrig
                  setacl write_schema/;
    # is the user command in the above list?
    if (grep $_ eq $command, @cmdlist) {
    ...

The admin type commands are typically global in nature, meaning, that a single Admin group is granted permission for these commands.  Stream creation has a more granular scope allowing different groups to control their development process and stream management capabilities.

Simply list the streams in the trigger that only Admins have the ability to “manage.”  By default, other streams not listed are managed by the development teams themselves.  There are 2 sections in the trigger to set this up depending upon the commands to control.

Restricts: lock, unlock, chstream, incl, excl, incldo

    $admin_stream{"replace_with_admin_stream"} = 1;
    $admin_stream{"replace_with_admin_stream"} = 1;
    $admin_stream{"replace_with_admin_stream"} = 1;
    ...

Restricts: mkstream, mkws

    $basis_stream_deny{"replace_with_basis_stream_to_deny"} = 1;
    $basis_stream_deny{"replace_with_basis_stream_to_deny"} = 1;
    $basis_stream_deny{"replace_with_basis_stream_to_deny"} = 1;
    ...

Inside the trigger logic, each command is evaluated and will allow the operation to complete or not.

For example, the following section validates the “mkstream” command:

if ($command eq "mkstream") {
...
 # only a user listed as an administrator can create a new stream
 # based on an existing stream in the "basis_stream_deny" list
 if ( defined($basis_stream_deny{$stream2}) and `$::AccuRev ismember $principal "$admingrp"` == 0 ) {
   print TIO "Basing a new stream on existing stream '$stream2' disallowed:\n";
   print TIO "server_admin_trig: You are not in the $admingrp group.\n";
   close TIO;
   exit(1);
  }
}

There are other facilities in AccuRev to control the process and workflow too. Stream Locks grant users/groups the ability to promote to and from streams and Access Control Lists (ACLs) grant access to entire depots and subhierarchies.  Setting up these security measures combined with the Admin Trigger provide your organization with the flexible and granular security model it needs for the optimum development process.

Drop me a note and let me know the creative ways you’re using the Server Admin Trigger.

AccuRev CTO, Damon Poole, is a guest speaker for an Agile webinar series, Agile 101: Laying the Foundation for Agile Development, presented by VersionOne.

Damon will be presenting this Wednesday on the following topic, and again on April 1 to discuss The Algorithms and Architecture of Agile vs. Traditional Software Development.

Breaking the Major Release Habit
March 11th
12:00 – 1:00pm EST

Are you frustrated with your release schedule? Are you considering moving to more frequent releases or wondering if there are ways to tweak your release schedule? Although the traditional pattern for software releases is to have a major release every 6-12 months or more and minor releases in between, there are many other release patterns to choose from. This session will examine the various factors and options involved, discuss their pros and cons, use examples drawn from a wide variety of companies, and give you the information you need to make an informed choice and then implement it.

Register

AccuRev is co-presenting a one-day seminar on Agile Development Transformation:

“Mitigating Risk with Agile Development: Great Software, Great Business Results.”

Where:10 Maguire Rd, Bldg. 1, Lexington, MA
When: Thursday, February 26, 9:30 – 3:30

Details and registration

Speakers: Rich Mironov (Enthiosys); Johnny Scarborough (GlobalLogic); Damon Poole (AccuRev)
Cost: $50 for qualified registrants ( List price $595 ). Seating limited to 50 attendees.

This one-day session will include detailed presentations, interactive exercises and open discussion on:

• Agile development approaches including distributed agile methods, the history of agile, and agile manifesto
• A detailed walk-through of Scrum, one agile approach
• The organizational changes required for successful agile adoption: executive commitment, cross-functional teams, and coaching
• Hear first-hand from one of your peers about how to bring about agile adoption and improved results.
• Roadmaps, releases, iterations and the iron triangle
• Business drivers, business value and customer collaboration approaches
• How to evaluate technologies when adopting agile

This seminar is intended for CTOs, Vice Presidents & Directors of:

• Software Development or Engineering
• Product Management
• Business Units

All attendees will receive free copies of two new books:

Scaling Software Agility: Best Practices for Large Enterprises by Dean Leffingwell
The Art of Product Management by Rich Mironov

More Agile resources

AccuRev is hosting a free Webinar on “Emerging SCM Best Practices for Agile Development” on Thursday, February 5 from 1:00 – 2:00 PM EST. The webinar will introduce the unique demands that agile processes place on legacy SCM tools and ways to build and automate an efficient Agile development process. Damon Poole, AccuRev CTO, and Uttam Narsu will be presenting.

Mr. Narsu, an industry expert on software configuration management (SCM) best practices and former Forrester/Giga analyst, will discuss the most important aspects to consider when applying SCM best practices to an agile world.  Mr. Poole will discuss how Multi-stage Continuous Integration can solve some of the underlying impediments to a successful Agile development environment.

“SCM is critical to Agile success. If your SCM tools don’t provide integrated and seamless support for Agile, you won’t get widespread adoption of Agile development,” said Narsu. “Development teams require brutally efficient SCM tools, but the tools must still be issue-based, must have support for flexible process models, and enable efficient branching, merging, and refactoring. The SCM tool should also provide private workspaces and assist continuous integration.”

Mr. Narsu has worked with hundreds of clients using wildly differing software development environments. While clients who succeed with agile development have strong SCM practices, the best practiced agile SCM: transparently integrated, flexible, and collaborative.

Specific topics discussed will include:

• Typical SCM obstacles to Agile success and how to avoid them;
• Key Agile Process requirements for SCM products and specific use case scenarios;
• Challenges with continuous integration, and how Multi-stage Continuous Integration delivers value and how to adopt it today; and
• Key SCM metrics for delivering on Agile development goals.

Register here for this free webinar taking place on Thursday, February 5th from 1:00-2:00 EST: Emerging SCM Best Practices for Agile Development.

by Lorne Cooper, CEO, AccuRev 

A Top 25 WordPress Blog Post of the Day

In Part 1, we looked at the sins of Building a Weak Team and Ignoring the Future.  Once you’re committed to hiring talent and building a forward-looking process, there are two more sins to avoid.

Sin #6: A Long Tail After Development is Complete

We constantly see organizations where the time from “requirements-in” to “feature freeze” is half the time to “solutions-out.”  That 50% “tail” on development provides no direct value to customers, but is there because of limitations in our development process.

Sometimes that tail is twice the length of the initial development. 

Ironically, while the long heavy tail is grown to protect the organization from delivering bad product, it eventually decreases response time until it becomes an instrument in the organization’s death.

But many industries have a long tail, don’t they?  Surely it doesn’t hurt the construction of bridges that the plans are completed long before the bridge is built.

For forty years people have tried to make analogies between software development and more mature engineering disciplines, say, like building buggy whips.  The designer of the buggy whip might make a sample or two to try it out on his children, then send it off to the factory, where a slew of Industrial Buggy Whip Engineers would spend months figuring out how to sew the damn things, buying up horse-hair, and hiring immigrants to sit at sewing machines. A year or so late, Pony Express delivers the new buggy whip to general stores all over the Grand Trunk Railway line. 

We can all certainly learn a lot from that.

Now try telling your customer that the problem she’s been working around is well understood, and in fact fixed, but she’s not going to get it for three months.  She’ll be looking for alternative suppliers before the screen door has hit your butt. 

Everything moves too fast to use that tired old process.  Frankly, I use to blame computers.  Now I blame the Internet. 

When you take a hard look at the time it takes to get completed product out the door, you’ll probably find it dominated by two things: testing, and fixing bugs that the testers find.  Both stem from the problem that you’re not doing enough testing during the development process, and you’re trying to test too much at one time.

I’m no fan of Hawthorn-effects, or what the politically incorrect might call gimmicks, like pair programming or stand-up meetings, but projects that use an Agile development with automated regression test processes are certainly doing more right than wrong. 

Is it really so hard to manage by tasks/issues, normalize them to a small unit of work, merge in just one set of changes into each build, and test just that perturbation?  What if you knew you would consistently get better code out the back of the process, and therefore make your organization more responsive to your customers?

Your development process must minimize latency, instead of trying to maximize bandwidth.  Don’t miss the changes in requirements and environments that produce opportunities for better products, or sooner or later you’ll end up locked in a suicide pact with a project no one wants.

Sin #7: Complacency

Calais’s stone walls were so thick the city had no fear of invaders, until they showed up with cannons.  Napoleon’s Old Guard was undefeated in battle after battle, until they were cut down by Wellington at Waterloo.  Big Blue’s OS/2 would crush little Microsoft’s Windows product.  Big Microsoft’s Live Search would crush little Google’s search engine. 

Hubris on ice will leave you with a heck of a hangover.

That brilliant team using that wonderful process you put together last year, is just a few months of complacency away from dysfunction.  People move and, thanks to the 13^th Amendment to the US Constitution, may leave their jobs and there’s little you can do about it.  Environments change and architectures become outdated.  The marketplace changes and requirements change with them.

One VP Engineering I know told me he asked his off-shore application development partner to give him 110%.  They did, but it turned out to be turnover.

Your process needs to be constantly monitored and upgraded, because everything in the process is changing.  If your offshore partner has high turnover, their part of the process might require more code inspection and knowledge transfer.  If your biggest customer can’t use your product until it supports their new environment, you need to put a special team on building a custom version of the product, and then merge that back into main development.  If you have to replace an old veteran with a rookie, then you might create new integration points and a more rigorous test protocol.

Recognize that everything will change, and the assumptions under which you built your last success are no longer valid.  Make small process changes early and often, or in the long run the big changes will be made by your replacement.

Stirring Conclusion

Managing a software development organization is a very challenging task, as tough as playing a violin with your toes, and less fun. 

I don’t mean to trivialize important management functions such as customer management, budget and project tracking, running a good meeting, and designing effective compensation structures. And by all means, hiring someone to spend their time doing all of that.

But that won’t protect you from the seven deadly sins.  Only you can do that.

I’ll finish with the Four Virtues, which are the antidote to the Seven Sins:

1. Hire great people or don’t hire.
2. Plan to build a product that will have a long life.
3. Keep your process nimble, and deliver less, faster.
4. Keep improving everything.

If you listen closely, you can almost hear the soft hum of thousands of well-intended voices all intoning the mystical phrase “Agile Development” like a magical mantra that will make everything faster, better and appear more attractive.  This buzz word is coming from managers and their bosses, from PMs and VPs and CMs (Configuration Managers) and other folks with 2-letter title abbreviations, from developers and testers and even the customers.   “We must be Agile!”– so they say.

As you may have noticed, if you repeat any word or phrase long enough, it tends to lose all meaning.  Unfortunately this seems to be the case with concept of Agile Development. 

I once attended a meeting wherein a VP announced that we were going to do agile development “as of today.” There was a lot of cheering and a lot of smiling and a few hands were shaken.  And at the very back of the room, there were a few of us that sat there quietly trying mightily to conceal our shock/disbelief/cynicism and sheer apprehension at the thought of what was about to happen to us. 

You see – Agile development is more than just throwing smaller chunks of code into Production faster.  It takes planning, involvement, a solid architecture, good supporting tools – in short, A WHOLE LOT OF WORK – to make agile processes really work for you.  You can’t reap the reward without doing the work first– and if you try, all you’re going to wind up with is a great, big mess. (Not to mention a staff with their updated resumes out on DICE)

While this post is written a bit tongue-in-cheek, the message is serious.  If you want to be agile, make an investment in the process:

1)      Know your code architecture:  Having all 73,000 files in version control is not the same as KNOWING the architecture of your code. You can’t be truly agile if you don’t know the inter-dependencies of your own code. 

2)      Know your end-users wants vs. needs: Actively involve the end-users in the release scope.  This is A LOT harder than it sounds.  It takes a good relationship with the end users to separate out their desires from their actual needs, and balance the content of the releases across the two.  Building this relationship is a fundamental component of agility.

3)      Implement Tools that support Agile methods:  There is nothing agile about merging branches of code all over kingdom-come.  There is nothing agile about having to manually determine what files changed since last Friday at noon, or depending on checksum to figure it out.  Choose your tools wisely, implement them appropriately for your individual situation, and enforce the process globally across all groups, management levels and situations…and do so knowing that everything is subject to change without notice.

I highly recommend AccuRev to support agile development methodologies.  It provides a level of flexibility that I’ve never encountered in any other tool, while still enforcing process through an indelible history of every event, and user defined process criteria.

AccuRev is the ideal tool for distributed development teams, with fast remote updates, the option of full or partial updates to the development workspace, and flexible, developer-defined and controlled sharing of in-process work.

I’ve setup a lot of projects using a lot of different software configuration management tools, and AccuRev is by far and away, my favorite choice for a SCM tool – particularly when supporting agile processes.

In closing, here are some words of wisdom from an old-hat Configuration Manager:

1)      If they tell you, “Just load the CM tool on the development server for now.  We’ll find you a permanent server later” – DON’T fall for it.

2)      When a prospective employee describes their environment as “dynamic” just know in advance that’s a euphemism for “chaos.”

3)      There is no such thing as a “Planned Emergency.”

4)      If your manager says, “We’re implementing agile methodologies, and we’re buying ClearCase, because it’s the best,”….well, in that case, I’ll be seeing your updated resume on DICE…

Fran Schmidt is a veteran CM, who’s survived over a decade in the Software Configuration Management field through a combination of good humor, constant education on the newest technologies, and sheer stubbornness.

A long, long time ago I started life (okay college life) as an Engineer. Electrical. Analog. For having grown up with always having a computer in my room I was about as far away from the digital world I could get and still be technical. Computing was easy. Oh pascal, if only you could see me now. Those were wild days, staying up until all hours of the night sitting in a basement computer lab trying to finish an assignment. When I wasn’t at classes or getting into a variety of trouble, I would spend time with my roommate, a Mac user. He’d mock my Amiga; “Do you really need two buttons?” I’d mock the Mac (pre-classic) “Hey I’ve got 4096 colors” but we’d agree on one thing, both were better than that loser IBM machine two dorm rooms down the hall.

Then came real life, I fell into the digital world, and found myself back in school getting a more formal digital education. I found myself in Design Patterns, listening to people around me furiously debating the merits and needs of singletons, factories and observers.

Then came Java, and the arguments continued. Performance, idiosyncracies of langauge (both C++ and Java) etc., etc.

Then came Open Source. Don’t get me started with open source. If the OS mafia reads anything I write they’ll try to shutdown our website.

Now we have Agile. And Lean Development. And Continuous Integration (CI). I’ll lump them together because they are the current arguments I hear today. Yeah, arguments. Developers become elevated and yell out “You’re not really agile!” I read books on what it means to be Lean, and how to identify if you are Lean. I find CI writers muttering under their breath that any build process that occurs hourly isn’t truly CI.

They’ve lost focus. Or more importantly their focus should not be your focus.

I wasn’t hired because I was a Design Patterns zealot. I wasn’t hired because there was a need for an expert on Lean Development. I wasn’t hired to support code in the OS movement. I was hired to produce a product. Our customers want our product, they want it yesterday, both features and fixes.

So what’s the best way to deliver value to the company and customer? To be honest, I may not know the best way, but I know we will do our best to do whatever it takes. I and my coworkers strive to find reasonable solutions in a reasonable time.

But the zealots cry out and tell you what you are doing wrong, not doing right. What they don’t understand is that if they want to be heard they have to show the value in the changes. You need to give the agile concepts breathing room. This is what really encourages adoption. Agile zealots need to understand why there is resistance, understand people’s concerns and address them, and believe me they will embrace the positive elements.

On the other side, why listen to these zealots? Because if you can get past the worship and hype, there is value in what they describe. If you listen, you can figure out what elements will be useful.

Agile really is an evolutionary model. Agile isn’t about anarchy or cowboy developers, but about adopting processes that improve on process management. Agile really helps break logjams where problems seemed intractable or a great unknown, places where development has struggled for decades over thinking problems only to have their solutions fall flat and disappoint customers.

Lean Development has a great focus on continuous refinement. Focus on problem areas and work to make it a little better. After you’ve done this take another look and make it a little better somewhere else. Very quickly the improvements add up.

And Continuous Integration keeps simple problems simple, resolving them in a timely manner. CI keeps developers from sitting on problems further compounding integration, testing and deployment.

There are values that are worth a look in all of these movements. Take what they give you, find the value and use it. Continuous Integration makes your life better. Problems are found (and hopefully fixed) when they happen, not weeks later. You are getting continuous feedback on the health of your project, not monthly or weekly feedback. If your life is better you can spend more time providing value to your company and your customers. If you deal with less pain you have more time to solve the problems you should be solving. And adding the features that your customers are waiting for and wanted yesterday.

What are you waiting for? Don’t be discouraged thinking you aren’t using CI ‘correctly’. Take elements and make them work for you, and the value will come quickly.

Streams are the workhorse of an AccuRev depot. Without streams, everyone would commit code to a single “bucket.” As you know, streams are connected together in a hierarchy that defines a promotion-based workflow. For mainline development, a workflow can be as simple as:

Dev > Test > Prod

or, sophisticated like:

Dev > UnitTest > Int > LoadTest > FlightTest > Prod

The Pattern

While streams are always used for organizing the core workflow, they can also be extended to logically organize developer tasks using a “stream-per-task” paradigm. The concept is simple: create a new stream for every task that one or more developers plan to work on. Name the stream logically, as in “Feature X” or “BugFix Y”. The developers then create their workspaces from this stream. This pattern works exceptionally well with agile development groups where work is broken down into tasks.

The Advantages

In theory, the task stream is just another stream and promotion step on the way to delivering changes to the core workflow (see adjacent picture). Though, in practice, the advantages are numerous:

• Organize development activity by feature or fix
• Visually see all active projects (removed task streams are implicitly finished)
• Visually see and manage who is working on any given project
• Develop individual projects against latest mainline without forced committing to the core (relies on inheritance and significantly reduces rollback)
• Control access to projects by user or group using stream locks
• Retarget entire projects and teams with a single drag-n-drop

The Critics

No suggestion is without its critics. Let’s address a few of the common claims. Claim: adding a new stream creates an unnecessary promotion step. Response: use a pass-through stream instead! Claim: creating a stream-per-task will create 10′s, or 100′s of streams. Response: Yes, but each stream is a 120-byte (yes, byte) record of meta data on the server so space consumption is negligible; the GUI ‘zoom’ feature makes navigating thousands of streams manageable; remove each stream when the project is finished; and using a good naming convention goes a long way. Claim: Creating streams is yet-another-step for someone; who will do it? Response: it depends; Sometimes savvy project managers enter them in AccuRev; other times it’s the release manager(s) who take this on as a routine task; but I say, empower the developers! Why not have the dev leads or project-oriented developers do the task — it’s sooo simple and can be controlled with locks, ACLs, or basic triggers.

For those critics still remaining who need proof, I’ve provided an illustrated stream hierarchy where everyone contributes to a single bucket. Compare this picture with the one above…. and I’m sure you’ll be convinced. In short, the drawbacks of everyone committing to the same bucket include:

• Unable to collaborate in isolation
• Requires reverting (unmerging) features who’s release has been postponed
• Limited visibility to active projects; requires strict wspace naming; hard to enforce
• Retargeting project work requires multiple workspace reparenting
• Unable to control per-project delivery or access with locks or ACLs

Concrete Example

With the pros and cons out of the way, lets show a concrete example. For both discussions above, I intentionally created stream structures in abstract form using generic names and a layout that is comparable. But lets get real. Here is a stream structure that represents a more realistic environment and even includes elements from a previous blog post on using the pass-through stream. Isn’t it great to see how this stream-per-task paradigm helps methodically organize activity across the entire depot!

Are there any other pros or cons to this approach? Does anyone have examples of using this pattern in their own implementation of AccuRev?

/happy tasking/ – dave