Archive for the ‘change packages’ category

From Merge Hell to Merge Master

March 23rd, 2011 by jsherwood

Are you frequently called on to perform the daunting task of ‘the merge’? Does tension mount and do groans grow louder when it’s time for a merge?  Are only a select few are willing to stand up and do the work?

Maybe these stages describe your merge process:

  1. Denial: It’s never going to work (in a timely manner).
  2. Anger: Why do I have to do the merge?
  3. Bargaining: I’ll work on maintenance and support if I don’t have to do the merge.
  4. Depression: How many compiler errors? Why won’t the jsp work with firefox?
  5. Acceptance: Not really, it’s compiling and (mostly) passing tests.

Before we try to get you through the 5 stages of merge, let’s a take a look at divergence, the cause of a complicated merge.

Converging Divergence

There are lots of reasons for divergence, and as many for merge product lines back together.

Maybe you’re delivering functionality to specific customers, creating customization and personalization of websites, internationalizing your product, even moving from Windows to Linux platforms. So what happens? These areas diverge, developers refactor code, and the product lines look very, very different over time. Sure, you try to propagate changes across the product lines, but each implementation is slightly different. Eventually you realize this is a maintenance nightmare, and some shared areas should be brought back in line (if not the whole product line moving to a single source).

Now comes the hard part. A select few of your developers have to start piecing the code back together. Determining what changes are acceptable, reviewing the functionality, and hopefully getting the appropriate test coverage to verify that the desired changes have been brought back into a single (or at least cleaner) product line. Here’s where tools really shine (or falter).

Now, I consider myself pretty experienced in the merge department. I’ve worked on a number of merges of different sizes, dealt with multiple languages (English/Italian), and even migrated to different platforms (Moving from Windows to Macintosh).  Sometimes, I found, tools were a complete hindrance- I remember back in the day, using PVCS or CVS was every man for himself. You just got it working, never mind worrying about who made what code decision, and hopefully extensive tests identified the problems. More modern tools, AccuRev in particular, give you another dimension with merge that I find to be an essential.

The Merge Master

AccuRev version tracking, and the changes it tracks (content, name, removal) really help in hunting down the straightforward changes that can result in truly subtle changes in product behavior. Simply knowing that files have been removed in one product, and merging those changes over with a click of a button can eliminate behavior that is hard to determine, especially when the developer doing the merge had a reasonable belief that the files may exist.

Even better, large or midsize merges, can be easily reviewed, grouped into areas of the code and dealt with in a manageable manner. I performed a merge that merged internationalization changes into a product at AccuRev. There were over 7000 files that were altered, added or removed that needed to be managed.

Simply grouping the files by directory structure quickly identified which were doc, help, client, java, etc. Then these areas could be dealt with by developers in the specific areas. Further, because AccuRev performs simple merges quickly, I was able to perform the initial merges in these areas without having a strong need to understand them, and leave the complex changes to those who specialized in particular areas. This further reduced the amount of effort (and pain) that other team members experienced.

In this 7000+ file example, only about 400 files actually required more specialized investigation. And of the 400, only about 50 had truly complex changes where both sides of the merge made what looked like contrary decisions. Here is another place where AccuRev shines. By looking at the changes made at different points in each of the product lines, and working with change packages that described what the developers where trying to accomplish, I was able to make intelligent decisions about how the code should integrate, instead of just picking ‘he who changed it last’.

The two product lines in this case had diverged for about 9 months, and had about 10 developers making changes in each product line. Even with the amount of changes that occurred over this time period, it only took a single developer about 2 weeks time to bring the two product lines together. It took only a couple of days to complete the initial code process, walking through the changes and picking (via the AccuRev GUI) which of the changes to take, and editing inline where it was obvious.

During the first week, we were able to finish digging around developer changes via the AccuRev version browser in order to help resolve initial compilation failures. This work brought the product to the point where it was compiling on multiple platforms (which is usually even a problem with nightly builds). After another week of going through the validation and user tests, the merge was stable enough to be considered the baseline for the next release. Of course development had already occurred during the two weeks that the merge process took, but with the version tracking AccuRev performs with merges, it became almost trivial to bring in the new changes- within 1 day they were validated and ready to be the new baseline.

Sounds like acceptance to me.

Agile Teams Move Quickly… Make Sure Your User Stories Can Too!

January 27th, 2011 by Bob DeMaria

It occurred to me recently how much emphasis is put on the need for an Agile Project Management tool as you begin to adopt and expand the use of Agile in your development environment.  While many, including myself, would argue that using post-it notes and physical story boards are the best ways to start and Agile pilot team (since I believe it instills good habits, like circling the team around a common goal) there is certainly a place for Agile Project Management as you begin to expand Agile within a larger organization.  Agile Project Management provides visibility across the development lifecycle and into teams spanning multiple time zones, allowing for a big picture view of the project.

User Stories are the Key

Driving development through the use of User Stories is often considered a determining characteristic of whether a shop is considered Agile or not.  And while I won’t go into the details here, it is important to have a ‘place’ to manage those user stories in a backlog.   It’s also important to be able to see and report which iteration the user story is assigned to, and the current status of a particular story.  Has it been scheduled for an iteration?  Have we started development on it?  Who’s working on it? Etc…

Understanding the state and status of user stories that are in development is crucial to accurately reporting the current state of an iteration as a whole.  If you are going to use burndown charts to know how many hours of work are left for an iteration, or a burnup chart to determine how many points are left to be accepted for an iteration, you are going to need accurate information.  Accurate data comes from updating user stories and tasks on a timely basis so you always have the freshest information.

How Accurate is Your Information?

One aspect of user stories that is often overlooked is the linkage between the user stories within an active iteration and the location of the actual code changes for those particular stories.  If you only have an accurate view of the stories from a development standpoint, but you have no idea where the code for that story is, then all of the information in your project management tool contains incomplete data.

Under more traditional development methodologies, the time to ‘check’ the linkage between closed issues and code that was changed in order to complete said closed issues happens on an infrequent basis.  Checking the linkage usually occurs toward the end of a long development cycle, when teams are getting close to a release point.  If we are executing on a 6 month release cycle, it probably won’t occur until 4 months in.

Identifying the status and locating the code for user stories happens weekly on an Agile team, regardless of the iteration length.  This requires you to have quick access to accurate information to track those stories from a development perspective.

Accurate Information… Quickly

In many traditional SCM systems, developers must manually indicate the linkage between the code and the story it is associated with as the code is checked in. At the end of the iteration, teams must determine what stories are fully completed and what stories are only partially done, and need to be retargeted to the next sprint.

Traceability and visibility into each story is necessary to see where code changes are located. The ability to easily associate code modifications with a user story will eliminate error prone manual linkage problems. In AccuRev, this information is stored in a Change Package, or a deep linkage between the user story that is related to project management and code. Change Packages can be tracked through the development cycle.  This gives developers visibility into the status of an individual user story without additional overhead, while providing traceability to Scrum Masters and Product Owners into the status of an iteration with increased accuracy.

Agile and SCM: Helping You Get to “Done”

January 24th, 2011 by clucca

You spent the last 2 weeks working on 10 user stories for you sprint. Your team has completed 8 of the 10 user stories, and now it’s time to show a working demo for your sprint review. Sounds like a typical scenario right?

Here’s the problem, the 2 remaining user stories have partial work associated with them. That code is in your source control system, and in order to correctly demo or ship what was accomplished in your sprint, you’re going to have to do some work to create a “done” product. You will have to subtractively merge those changes, or put them on another branch, then re-test the application, redeploy.. check the changes again etc. There is still a lot of work to finished in getting this sprint to “done.”

One way to get around this is to associate user stories with code changes in your SCM tool.  AccuRev already creates a link between your user stories and code in the form of change packages. Creating that link between the user stories and your code is the first step. But using a powerful story-driven process out of this linkage is the key to “getting to done” in an Agile environment.

Your user story will move through different stages as you move through your process. One user story may have these states:

chart Agile and SCM: Helping You Get to DoneDEV- Currently in progress with dev team, or “coding”
QA- Coding finished, ready for QA to test
UAT- QA finished ready for user acceptance testing
PROD- Tested by QA and users, ready to ship

However, while  user stories move through this process, the code doesn’t follow. Traditional SCM systems are designed around single branches and a long and lengthy merge processes. Fast paced Agile environments expose those limitations as development teams struggle to ship code to customers at the end of iterations.

During that iteration, testers may want to test completed user stories, but need a stable configuration to do so. If you’re using a traditional waterfall branching and merging based SCM tool, you may end up with some user stories ready for QA, and some still in the DEV stage, creating a poor testing environment and broken builds. As a result, developers often delay committing code so changes can be implemented, tested, and passed to QA before they integrate.

During your retrospective of the previous iteration, you may decide that parts of your process need modification. For example, you may decide to add a code review process right before testing. Your SCM system has to be flexible to enable this new step without time consuming tasks, like re-writing scripts.

SCM and an Agile Process

The trick is to have an SCM system that can help you manage and enforce an Agile process. Code should be able to follow the same process user stories follow. If a user story is in QA, all of the code needed to test that issue should reside in a QA configuration. The same goes for DEV, UAT, PROD, or any other process that is part of your environment.

A stream structure is an easy way to  process change – streams can be added to adjust your process. In a few clicks, you can add a code review between a DEV, and QA step.

Agile and SCM: Avoiding Agile Merge Hell

As you start to scale Agile, code and user stories have to be merged more often. Sometimes changes my flow from one organization to another. This means that you will need to take code from one team, merge, integrate and test those changes with everyone. Each team needs to be able to work on their own schedule, this means that if multiple teams want to work on different sized iterations they can. In addition they can deliver changes as they need to on a regular basis, independent of the the other teams.

The problem here is that to do this in a traditional SCM system, you would have to merge these code changes daily for them to be of any use. You also still need to keep visibility into what stories are shared between teams, because delivering changes from user stories that are not completed in a sprint would be disastrous.

Typically, the type of configuration people use for this is a single baseline or “trunk” methodology, where all changes from each team are delivered to trunk and pulled from trunk as their iterations are completed.

Working within an Agile context, your teams will have to deal with these branching and merging issues. But there are other problems that can happen when you use this methodology:

  1. Delivering 2 weeks worth of changes only causes isolation among teams.
  2. It’s too hard to pick out each user story as it completes from our codebase.
  3. Figuring out the dependencies of those user stories is complex.
  4. Being able to identify what changes came from each branch is impossible.

This “baseline pollution” is not scale-able. You can get around this by using a development hierarchy, and manage the relationship of dependencies between branches. This could also include process steps such as integration, quality assurance, and code reviews.  A separate code configuration can be used for each step and user stories could simply be drag and dropped between each team, state or branch instead of a merge.

Doing this will increase code stability. As a completed user story is pushed from one stage to the next, the particular change, as well as the system as a whole, reaches a higher level of maturity. While many traditional SCM tools do not easily support or surface a development hierarchy, AccuRev supports the creation of a hierarchy, gives visibility into the changes at each stage, and enables straightforward merging between stages.