Posts Tagged ‘waterfall’

Agile and SCM: Helping You Get to “Done”

January 24th, 2011 by clucca

You spent the last 2 weeks working on 10 user stories for you sprint. Your team has completed 8 of the 10 user stories, and now it’s time to show a working demo for your sprint review. Sounds like a typical scenario right?

Here’s the problem, the 2 remaining user stories have partial work associated with them. That code is in your source control system, and in order to correctly demo or ship what was accomplished in your sprint, you’re going to have to do some work to create a “done” product. You will have to subtractively merge those changes, or put them on another branch, then re-test the application, redeploy.. check the changes again etc. There is still a lot of work to finished in getting this sprint to “done.”

One way to get around this is to associate user stories with code changes in your SCM tool.  AccuRev already creates a link between your user stories and code in the form of change packages. Creating that link between the user stories and your code is the first step. But using a powerful story-driven process out of this linkage is the key to “getting to done” in an Agile environment.

Your user story will move through different stages as you move through your process. One user story may have these states:

chart Agile and SCM: Helping You Get to DoneDEV- Currently in progress with dev team, or “coding”
QA- Coding finished, ready for QA to test
UAT- QA finished ready for user acceptance testing
PROD- Tested by QA and users, ready to ship

However, while  user stories move through this process, the code doesn’t follow. Traditional SCM systems are designed around single branches and a long and lengthy merge processes. Fast paced Agile environments expose those limitations as development teams struggle to ship code to customers at the end of iterations.

During that iteration, testers may want to test completed user stories, but need a stable configuration to do so. If you’re using a traditional waterfall branching and merging based SCM tool, you may end up with some user stories ready for QA, and some still in the DEV stage, creating a poor testing environment and broken builds. As a result, developers often delay committing code so changes can be implemented, tested, and passed to QA before they integrate.

During your retrospective of the previous iteration, you may decide that parts of your process need modification. For example, you may decide to add a code review process right before testing. Your SCM system has to be flexible to enable this new step without time consuming tasks, like re-writing scripts.

SCM and an Agile Process

The trick is to have an SCM system that can help you manage and enforce an Agile process. Code should be able to follow the same process user stories follow. If a user story is in QA, all of the code needed to test that issue should reside in a QA configuration. The same goes for DEV, UAT, PROD, or any other process that is part of your environment.

A stream structure is an easy way to  process change – streams can be added to adjust your process. In a few clicks, you can add a code review between a DEV, and QA step.

Agile and SCM: Avoiding Agile Merge Hell

As you start to scale Agile, code and user stories have to be merged more often. Sometimes changes my flow from one organization to another. This means that you will need to take code from one team, merge, integrate and test those changes with everyone. Each team needs to be able to work on their own schedule, this means that if multiple teams want to work on different sized iterations they can. In addition they can deliver changes as they need to on a regular basis, independent of the the other teams.

The problem here is that to do this in a traditional SCM system, you would have to merge these code changes daily for them to be of any use. You also still need to keep visibility into what stories are shared between teams, because delivering changes from user stories that are not completed in a sprint would be disastrous.

Typically, the type of configuration people use for this is a single baseline or “trunk” methodology, where all changes from each team are delivered to trunk and pulled from trunk as their iterations are completed.

Working within an Agile context, your teams will have to deal with these branching and merging issues. But there are other problems that can happen when you use this methodology:

  1. Delivering 2 weeks worth of changes only causes isolation among teams.
  2. It’s too hard to pick out each user story as it completes from our codebase.
  3. Figuring out the dependencies of those user stories is complex.
  4. Being able to identify what changes came from each branch is impossible.

This “baseline pollution” is not scale-able. You can get around this by using a development hierarchy, and manage the relationship of dependencies between branches. This could also include process steps such as integration, quality assurance, and code reviews.  A separate code configuration can be used for each step and user stories could simply be drag and dropped between each team, state or branch instead of a merge.

Doing this will increase code stability. As a completed user story is pushed from one stage to the next, the particular change, as well as the system as a whole, reaches a higher level of maturity. While many traditional SCM tools do not easily support or surface a development hierarchy, AccuRev supports the creation of a hierarchy, gives visibility into the changes at each stage, and enables straightforward merging between stages.

Stable Builds as Stories are Completed

June 18th, 2010 by damonpoole

One of the questions I hear a lot from Agile teams is “how can we have a stable build to test stories as they are completed?” Often, the only time the build is stable is towards the end of the iteration. That then squeezes the QA folks or sometimes even has them testing the previous iteration’s stories in the current iteration. Hmm. That sounds a bit like our old friend Waterfall.

Usually, when the question is asked, the team is creating and automating unit tests and doing continuous integration. That’s not the issue. (If you are not doing those things, stop everything and implement these three practices immediately!) But let’s assume for the moment that you are doing these three practices.

The problem is getting a build which consists of only stories that are considered “complete,” but still need a bit more work done on them, such as exploratory testing, load testing, usability testing, demo for customer feedback, etc. You might say, “but you can’t do that until the end of the iteration.” Ok, but then you are back to the problem of a compressed QA cycle and having development either sit on their hands or move on to the next iteration, neither of which represent Agile thinking.

There’s a simple solution to this problem, which you may have “thrown out with the bath water.” Branches! Or, in AccuRev parlance, streams (though I’ll stick with branches for simplicity). Each process stage gets its own branch. That doesn’t mean that you can’t write code and tests at the same time, only that code can’t advance to the next branch until it meets the criteria of that branch, such as “coding is done” and “all tests are written and passing.” I know, you cringe at the thought of branching and merging, but that’s probably because you are thinking of branches that contain long-lived changes. We’re not going to do that here.

The idea presented here is to advance changes through the set of branches as quickly as is practical. As changes get propagated to each branch, CI is done against that branch.  If it succeeds, it gets promoted to the next branch. If it does not succeed, then that developer (possibly with help from her teammates) fixes the branch. This is similar to how stopping the line works in a modern lean manufacturing facility.

In AccuRev, this process is greatly simplified. You can create a set of streams instead of branches, and streams can model your exact process. Promoting work from stream to stream is a simple drag and drop operation.

Getting Stable Builds as Stories are Completed

In the diagram above, there are streams for each of the stages from “wip” to “done.” Since each stage is a stream, you can run continuous integration at each stage. You can also do builds, for instance from the “tested” and “done” streams to do things like exploratory testing. By definition, a build from the “done” stream only contains changes which are built; integrated together; have unit and other tests written, automated and passing; and whatever other criteria you have for “done.”

Blending Agile Practices Into Your Traditional Development Process

June 16th, 2010 by damonpoole

When talking about Agile development, people often ask me, “but why would I want to change to a new methodology when traditional development has made so much money and created so much value for so many people for such a long time? Why mess with success?” That’s a great question!

After hearing that question for the umpteenth time I finally realized something.  It isn’t traditional software development that has provided so much value! It is the software itself. Imagine calling a travel service prior to the advent of software. You call them up and say “I’d like 100 or so options for travelling from Boston to San Jose. I’ll also need a bunch of 3 or 4 star hotels in San Jose and mid-sized rental cars to choose from.  Could I get that in 5 seconds please?”

In today’s world, of course you just go to a site like Orbitz.  It isn’t traditional software that provides that value, it is the software itself.  It almost doesn’t matter what process you use to produce that software, the value is so high. When you are only competing against somebody that doesn’t have software, that’s one thing. When you are competing against other organizations that are also using software, then the exact process you use can make a big difference from a competitive standpoint.

It isn’t traditional software development that has provided so much value it is the software itself.

What do we mean by “traditional development?” After having been involved in the process of literally thousands of software development organizations, I can safely say that there isn’t really any such thing as “traditional development” or even “waterfall.”  That is, if you look at any two organizations developing software, there is no standard for what comprises traditional development.

In the end, organizations adopt individual practices that make sense for their circumstances. This same approach can be used when adopting “Agile.” There really isn’t any such thing as “Agile.” It is a combination of values and practices which can be adopted by any organization and does not require wholesale adoption in order to provide value. Yes, there are packaged solutions out there such as Scrum, XP, Kanban and others, and they do provide value.  But so do the individual practices.  So, instead of thinking about “how do I switch to this new thing,” consider Agile as a new set of tools that you can add to your existing process on a case by case basis. Worry about “being Agile” after you’ve gotten some experience under your belt with some of the practices.