Archive for June, 2010

Change Packages for Agile Workflow

June 24th, 2010 by damonpoole

User Story Based Engineering

In other blog posts I’ve talked about using Multi-stage Continuous Integration to scale Continuous Integration for use in multi-team environments and using streams or branches to model your workflow directly in your SCM tool. While both of these approaches work well and provide a lot of value, they can also present a new challenge: how to associate enhancement requests and defects, which can be lumped together using the term “work items,” with the changes that implement them?

If you are doing mainline development, life is pretty simple. On check-in enter the enhancement request ID or defect ID in the comments. Once that association is made, it is fairly straightforward to figure out which enhancements and defects have been checked in. On the other hand, the more people you have checking in to the same branch, the less likely it is to be stable, even if you are doing Continuous Integration. So, how can we maximize the benefits of using streams to represent process and integration stages and still have the ability to match work items to the work done to implement them?

Representing Process and Integration Stages with Change Packages

(Let me first point out that while I’m using the generic term “work item” here, in an Agile team one would most likely use the term “User Story.” Both apply equally well in this context, but I’ll stick with work item for now.)

There is an SCM concept that maps to a work item which can be tracked from place to place, and that is a “change package.” It is called a change package because it represents something which, once defined, can be applied to multiple places and tracked regardless of how many different places it has been propagated.

Change packages can be promoted individually or in groups from stream to stream. That way, as work is promoted from stream to stream you have a record of the content of each stream from a high level perspective instead of a files perspective. A list of files is not very meaningful or useful. On the other hand, change packages allow you to ask questions like “which user stories are in the stream that represents all user stories that are ‘done’,” without having to resort to mainline development.

Something’s A-Twitter in My Backing Stream

June 22nd, 2010 by amonty

One of the cornerstones of any successful organization is communication. On Agile teams, we often meet to share information, update one another on progress, to reflect on that progress and discuss how the process can be improved. These interactions, be they stand-up or retrospective meetings, provide this information at regular intervals. But what if you need “real-time”  information sharing?

Consider the concept of “one piece flow”, often treated as the holy grail of engineering process purists. Lean/Kanban fan boys live to talk about this at conferences, and in the isle outside my cube. The idea is that a single “workpiece” at a time moves through the workflow. In the software development process, this is rarely limited to a single item, but instead is throttled by WIP limits to minimize the bottlenecks in the process. For example, the number of items worked on by developers is limited so that the count of items designated “ready for test” is kept at a manageable level. As a tester on an agile team, I’d like to know as soon as an item is moved from the “wip/development” phase to the “test/validation” phase.

Consider the following basic stream structure -

Simple Stream Structure

In an ideal model, testers would be notified as soon as an issue is promoted from the WIP stream to the Validation stream. This model assumes that the project is utilizing AccuRev change packages to track work items as issues. Having a distinct stream for validation purposes simplifies the job of the tester. All issues that exist in this stream have been unit tested by developers, passed basic regression tests, and are ready to be explored, validated, and promoted to the next stage in the workflow.

So, how can this level of communication be achieved? AccuRev triggers! The rest of this post will demonstrate how the server post-promote trigger could be used to provide updates using arguably the king of “What’s Happening?” – Twitter.

Why use Twitter in your Backing Stream?

Why not? Sure, you could use e-mail. But who wants more e-mail? Tweet it, and let them aggregate the information for you. Team members can follow the account, or not. Pointy-haired managers that dream of pie charts and love visibility can subscribe to an RSS feed, that you don’t have to manage. Lastly, if you’re an agilista, you’re already hip and trendy, (and let’s face it, you probably  already tweeted about the awesome presentation on Lean that you saw at the latest Agile conference).

The Server Post-Promote Trigger

I am not going to go into the gory details of the AccuRev trigger system. Here’s what you need to know.

  1. The trigger can be written in whatever language you like. In this example, I use Ruby.
  2. The trigger needs to be registered with the AccuRev server, and associated with the depot for your software project.

For this exercise, I wrote a quick script called tweet_post_promote.rb. The first step is to register this script with AccuRev.

>accurev mktrig -p SoftwareProject server-post-promote-trig ~/dev/ruby/tweet_post_promote.rb

Once registered, the trigger will now fire for all promotes in the SoftwareProject depot. Let’s take a look at the contents of the script.

#!/usr/bin/env ruby

require ‘rubygems’

require ‘crack’

require ‘twitter’

ACCUREV_DIR = “/sandbox/amonty/accurev/”

ACCUREV_STORAGE = “#{ACCUREV_DIR}storage/”

ACCUREV_SITE_SLICE = “#{ACCUREV_STORAGE}site_slice/”

USER = ‘ValidationBot’

PASS = ‘password’


def load_trigger_data(file)

xml = ”

File.open(“#{ACCUREV_SITE_SLICE}#{file}”) do |f|

xml = f.read

end

Crack::XML.parse(xml)

end


def twitter_get_auth(user, pass)

httpauth = Twitter::HTTPAuth.new(user, pass)

Twitter::Base.new(httpauth)

end


def main()

hash = load_trigger_data ARGV[1]

if hash["triggerInput"]["stream1"] == “Validation” then

changePackageIssue = hash["triggerInput"]["changePackages"]["changePackageID"]

update_string = “Issue #{changePackageIssue} just promoted to stream #{hash["triggerInput"]["stream1"]}”

twitter = twitter_get_auth(USER, PASS)

twitter.update update_string

end

end

main if __FILE__ == $0

This script is very simple. It makes use of the Ruby gems crack and Twitter. Essentially, the script takes the XML trigger file (provided as ARGV[1]) and loads it into a hash. Then the script checks to see if the stream being promoted to is our Validation stream. If so, it uses the twitter gem to update the status on our ValidationBot account.

Something's a-Twitter in my backing stream

A more useful example is to provide a link to the AccuRev Web UI Issue screen.

Something's a-Twitter in My Backing Stream

This way, anyone subscribing to the feed can click on the link and open the issue in the AccuRev Web UI.

Something's A-Twitter in My Backing Stream

This is a small example of how AccuRev triggers can be used to increase communication on your team. Testers can follow the ValidationBot account and be notified via their favorite Twitter client whenever an issue is promoted to the validation stream and they need to begin work on it. This could obviously be extended to include additional information (actual file changes, for example). That is, of course, if you can fit it into 140 characters. :)

For more information on AccuRev triggers, please see the Administrator’s Guide.

Stable Builds as Stories are Completed

June 18th, 2010 by damonpoole

One of the questions I hear a lot from Agile teams is “how can we have a stable build to test stories as they are completed?” Often, the only time the build is stable is towards the end of the iteration. That then squeezes the QA folks or sometimes even has them testing the previous iteration’s stories in the current iteration. Hmm. That sounds a bit like our old friend Waterfall.

Usually, when the question is asked, the team is creating and automating unit tests and doing continuous integration. That’s not the issue. (If you are not doing those things, stop everything and implement these three practices immediately!) But let’s assume for the moment that you are doing these three practices.

The problem is getting a build which consists of only stories that are considered “complete,” but still need a bit more work done on them, such as exploratory testing, load testing, usability testing, demo for customer feedback, etc. You might say, “but you can’t do that until the end of the iteration.” Ok, but then you are back to the problem of a compressed QA cycle and having development either sit on their hands or move on to the next iteration, neither of which represent Agile thinking.

There’s a simple solution to this problem, which you may have “thrown out with the bath water.” Branches! Or, in AccuRev parlance, streams (though I’ll stick with branches for simplicity). Each process stage gets its own branch. That doesn’t mean that you can’t write code and tests at the same time, only that code can’t advance to the next branch until it meets the criteria of that branch, such as “coding is done” and “all tests are written and passing.” I know, you cringe at the thought of branching and merging, but that’s probably because you are thinking of branches that contain long-lived changes. We’re not going to do that here.

The idea presented here is to advance changes through the set of branches as quickly as is practical. As changes get propagated to each branch, CI is done against that branch.  If it succeeds, it gets promoted to the next branch. If it does not succeed, then that developer (possibly with help from her teammates) fixes the branch. This is similar to how stopping the line works in a modern lean manufacturing facility.

In AccuRev, this process is greatly simplified. You can create a set of streams instead of branches, and streams can model your exact process. Promoting work from stream to stream is a simple drag and drop operation.

Getting Stable Builds as Stories are Completed

In the diagram above, there are streams for each of the stages from “wip” to “done.” Since each stage is a stream, you can run continuous integration at each stage. You can also do builds, for instance from the “tested” and “done” streams to do things like exploratory testing. By definition, a build from the “done” stream only contains changes which are built; integrated together; have unit and other tests written, automated and passing; and whatever other criteria you have for “done.”