Archive for June, 2008

Build Management with AccuRev and Maven

June 25th, 2008 by matthew d. laudato

Build management is a long-overlooked area of software development. As is often the case with underserved areas, the open source community has several good solutions. Maven is among the most interesting and functional solutions for Java developers. I had the chance to take the AccuRev-Maven m2eclipse integration out for a test drive recently. This is a recently announced integration (read the press release here if you are interested, and see my video demo here).

The integration follows the pattern of other software configuration management (SCM) integrations with m2eclipse. A backend provider interface was written to support AccuRev, and the provider was integrated into the m2eclipse environment via the SCMHandler mechanism. I used m2eclipse in conjunction with AccuBridge for Eclipse (the AccuRev-provided Team plugin) to materialize Maven projects into a new AccuRev workspace from within Eclipse, and then did some basic Maven build and AccuRev SCM operations.

To get started, I needed to install m2eclipse from Sonatype. I used the Eclipse Update Manager to do this, by creating a new remote site in Update Manager for Sonatype (http://eclipse.org/m2e/). I already had the AccuBridge for Eclipse plugin installed, also obtained via Update Manager (http://www.accurev.com/download/eclipseupdate). After restarting Eclipse, I was ready to test.

In my test environment, I had an AccuRev stream called maven_int that I had pre-loaded with (of all things) the source and test code for the Maven integration itself. My goal was to create a new AccuRev workspace based on this stream from within Eclipse that was Maven- and AccuRev-aware. To do this, I used the Eclipse Import functionality and selected the “Checkout Maven Projects from SCM” option. Since I was using AccuRev, I chose the AccuRev provider from the drop down list box. Like other integrations with Maven, AccuRev supports a URL format for specifying where to obtain code. The URL lets you specify the AccuRev user and login credentials, as well as the exact depot and stream from which to import the code. I also chose the make workspace checkout option, which tells the plugin to create an actual AccuRev workspace in a specified location.

After pressing the Finish button on the Import wizard, m2eclipse called out to AccuRev, created a new AccuRev workspace, and created new Eclipse projects (based on whatever Maven POM files were found in the maven_int stream) containing the source code. I then associated the Eclipse projects with AccuRev using the Team Share option so that I would have full access to AccuRev functionality.

Since the projects were all Maven-based, building was done by right clicking on the POM file and selecting one of the build options. To convince myself that the AccuRev functionality worked, I did some edits to a few files, saved the changes, and then used the AccuRev promote option to version that file in the AccuRev repository.

Overall, the integration worked as I expected. The installation of the plugins was fast, it was easy to materialize Maven-based projects from AccuRev, and all of the important Maven and AccuRev functionality was available in the respective plugins. If you are an AccuRev user and want to use Maven, then the combination of m2eclipse and AccuRev integrated inside of the Eclipse environment is a practical desktop tool for managing your local builds.

Continuous Integration: Begin Again

June 12th, 2008 by jsherwood

We’ve been using CruiseControl productively for a number of years. First as a stop gap in ensuring build success on multiple operating systems, later as a quick check of tests. More recently we’ve added reporting, and tests as part of a larger memory analysis platform.

But, being a bunch of software developers means we are continuously craving something different, like switching to Haskell and using Ruby on Rails. Joking aside (and nobody panic) it is more that we look at areas of our build processes to identify the pain points, and what can be done to improve them. I mean, that’s how we provide value right? By continuously improving processes and features.

This time we took a look at things we don’t change often, but end up injecting a lot of pain into the process each time they are done. And these pain points become more obvious as we increase our agile usage, as they occur more frequently than they did in the past.

How often do you go through your “official” release process? In my past life it used to be this occurred every six months at best, a year or so on a bad year and a long time ago (in a far far away land) I had the opportunity to work on a project that lasted 7 years. Now granted I certainly wasn’t on it for seven years, people tended to drift on and off the project but it took seven years to ship. That was a long time ago.

Today we are looking at monthly cycles, or in the case of very specific changes even two week cycles. Those painful processes that were blotted out with 6 months between them now happen pretty frequently. We used to not bother with the problems, just moan a little and trudge through it. But agile changes that. Continuous Integration changes that. These processes should work, be easy to change, and adapt quickly. Anything that doesn’t adapt you want to drop as quickly as possible.

To do that we’ve started looking at new tools for our processes. Because new tools solve all your problems right? As soon as you started compiling in C++ from C all your problems went away right? Okay not really. Tools are nice, tools make a good impetus for change, but change solves your problems. Yes moving to C++ provides you with access to new functionality, but only until you actually implement and use exception handling in a structured manner do you really get value out of the tool.

Sometimes it’s felt that the tool is enough. I mean when you switched SCM products you kept your old processes. Sure, maybe they didn’t quite fit the new tool and you had to write a lot of scripts, but it was what you were used to. You may even be nodding your head right now. Nothing like shoe horning a process you created 5 years ago into a tool adopted last year. If you didn’t change your process how did the new tool provide value? Performance and scalability may get you some value, but new features and usage patterns get you a lot farther.

As part of examining our build and integration tools, we looked at processes. What areas of our process are not formal? Is there a specific person who knows the incantations to make it work? What happens when machines fail, do we have alternative machines that can take up the slack? Can we run multiple builds on the same machine? Multiple tests? How do we get and review our results? What are the top 3 pain points for each person involved?

Not only does asking these questions help us understand the process better, but it identifies the areas of change. And with these changes we can leverage both our existing tools as well as the new tools we’ve chosen to acquire. It helps our existing team, and it helps those who join us in the future. They’ll be more productive faster. We’ll have time to work on the features our customers want, and instead of just getting by we’ll be able to use forward looking technologies.

Agile Requirements Traceability with AccuRev and Rally

June 11th, 2008 by matthew d. laudato

Today, AccuRev and Rally announced our technology partnership. You can read the press release here. Since I was part of the engineering team that did the initial proof-of-concept integration between AccuRev and Rally, I thought I’d spend few moments writing about the integration and how it helps customers connect requirements managed in Rally with code changes managed in AccuRev.

First, for those of you who prefer the movie to the book, you can view a short video demonstration that I recorded here. It highlights the main functionality of the integration.

Now to the details. Rally provides agile project management, including story and defect tracking, to assist customers in managing the rapidly changing flow of requirements and issues that are common in Agile development processes. AccuRev provides innovative stream-based SCM and integrated issue tracking to enable software process automation within the SCM tool. The integration ties AccuRev and Rally together to tighten the loop between requirements and defects, and the code changes that developers perform in order to satisfy those requirements or fix the defects.

The integration, written in Ruby and Perl, consists of three parts.

1. The integration queries AccuWork, the integrated issue tracking system that is available with AccuRev, for all ’New’ defects. It then transfers these defects into Rally, and makes an annotation in a custom Rally field called ‘AccuWork’. This field stores the AccuWork issue ID number.

2. When developers working in AccuRev make a code change, they will promote that code to an AccuWork issue, and place the ID of the Rally defect (created in the first part of the integration) in the promote comment. The AccuRev server_post_promote trigger (written in Perl) then fires and executes another piece of Ruby code that parses the promote transaction and sends relevant information about the code change to the Rally defect specified in the promote comment. This information is entered into Rally automaticaly and shows up as a ‘Discussion’ entry on the defect. Currently the integration posts the names and version identifiers of the AccuRev files that changed, the user id of the developer who did the change, and the timestamp of the change.

3. Finally, when a developer or Product Owner using Rally sees the code change, they can set the status of the defect to ‘Fixed’. Ruby code can then be run automatically that looks for all ‘Fixed’ issues in Rally, and changes their status to ‘Fixed’ in AccuWork. It does this by looking for the custom field ‘AccuWork’ created in the first part of the integration, extracting the issue ID number, and formulating an update XML message to AccuWork instructing AccuWork to update the specified issue.

That’s pretty much it. Working in Ruby and using Rally’s Ruby REST API was very straight-forward, as was working with the AccuRev XML API for querying and updating AccuWork. The end result is an early stage integration that provides basic requirements traceability between issues in AccuWork and Rally, and the coding activities of developers. I hope that if anyone is interested they’ll view the video and let us know what other features they’d like to see, so we can add them to our backlog and to future iterations of this integration in engineering.