Posts Tagged ‘source code control’

« Older Entries
Newer Entries »

Continuous Integration: The temptation of the Antipattern

April 30th, 2008 by jsherwood
Share|

You’ve started using Continuous Integration (CI), but it’s not quite meeting your needs. You’ve started out simply enough, getting a build to happen on Linux or some other common environment. You’ve gotten comfortable enough to start using it in more and more places.

You hear the whispers of potential. Maybe they come from coworkers, maybe you hear them in your head, maybe you’re hearing the antipattern. Yeah he’s talking to you, and he’s very tempting. Go ahead, add more tests, he says. Automated documentation, yeah that will work. Run some performance and benchmarking. Run some reports it can take it he says. Worse yet, the antipattern is sitting in the cube next to you. The antipattern doesn’t like to check in his code until it is ‘just right’, and then it is a big chunk of messy merged globby spaghetti code I wouldn’t eat and your product shouldn’t either.

The antipattern is tempting. Even the “good book” of CI talks about being careful of the antipattern, and then goes on to say go ahead and add more to your CI process. Very very tempting.

Beware of the temptation! Don’t fall into the hands of the antipattern. It only results in disappointment. There are people out there just waiting to show that CI isn’t successful. They are waiting for the first real failed build, delayed delivery, or time spent working on a tool instead of the product to kill the usefulness.

I should know, because I’ve given into temptation. We had a good CI setup going, our build processes were running smoother, broken builds were being caught before they became critical. And then I added benchmarking. Worked great. We were getting additional reports on leaks and bottlenecks. And we had time to attack them and slowly make them less and less of a problem. We were being lean. But then I heard the whisper. Run reports it said. Build some doc it said. So I tried. I wanted automated reports, one less thing to do manually right? But I wanted them with every CI build. Now the builds were taking 1/2 hour to an hour. Not so bad. Until code started breaking. You see, at the same time that I was tempted so were others. And it was easy. Large chunks of code were being added, destabilizing builds. Tests were failing over and over until people stopped listening to them. There was no buffer between changes and the rest of the developers. Yeah, we were able to clean up the mess using AccuRev, but the CI damage had been done. Benchmarking fell by the wayside while I tried to get reporting done. And the build took days to stabilize even while we scraped the code clean of the problem areas. And this was when the antipattern was able to leverage our misstep. Leaks got into the code. Not enough to destabilze, but enough that product performance could suffer. Instead of catching it early we caught it late, and the cost was that much higher when we fixed the problems.

That was over 2 years ago, but I’m here to repent. I had forsaken the usefulness and message of CI. The temptation was to strong for me then, but I and our product have recovered. Non-critical automation is no longer a dream to be shared with CI, but as a post operation run more infrequently. Stability and Performance of our product our paramount and CI is helping us keep them in line. I have exorcised the antipattern and remain vigilant.

2 comments »

Posted in Agile, Best Practices

Tags:

When SCM meets Web 2.0 – Cool Widget at Orbitz

April 17th, 2008 by jtalbott
Share|

In my role as an AccuRev Systems Engineer, I have the welcome opportunity to visit many companies looking to find a better software configuration management (SCM) tool, as well as many companies who have already achieved that goal by choosing AccuRev. I run into various degrees of sophistication, from those that were previously versioning their source by creating .bak directories to those that have embraced modern capabilities and are truly stretching their SCM environment beyond anything previously achievable.

One of the coolest examples I’ve ever seen of the latter is at Orbitz. I’m sure you’re familiar with the travel giant, and they use AccuRev to manage the source code behind their heavily visited website. Their team is full of highly skilled and intelligent people and they are definitely one of the most advanced AccuRev shops I’ve had the pleasure of working with.

While visiting them recently, I was very impressed with a component of an intranet wiki they showed me. They were actually running RSS feeds from AccuRev streams! There are a number of practical applications for this kind of functionality, but first and foremost it allows you to keep teams up to date on changes being made in the codebase. So you can subscribe to various collaboration streams in your project, as well as perhaps “watching” other streams that you might have interest in. This can also provide support for a continuous code review process. Leads on a project can easily review the code that is created during the project by watching the stream that the team is working in. By doing this review over the lifetime of the project, they can more easily give feedback earlier in the process. And since the feeds are simply aggregated as they arrive, they can decide when the best time to check out the changes are.

How is this done? Well, with AccuRev’s command-line it’s actually a lot easier than you might think. Their RSS feeds are generated from a Rails app they have written for internal use. Getting the initial feed is quite easy. The user’s feed reader hits a URL that includes the stream they are watching. This URL is parsed by Rails and sent to a Rails controller that queries for the last 20 transactions in that stream:


[snip]
Parameters: {“format”=>”atom”, “action”=>”hist”, “id”=>”orbitz-api-foo-ci”, “controller”=>”accurev”}
Accurev command (1.226238): accurev ‘show’ ‘streams’ ‘-fix’ ‘-s’ ‘orbitz-api-foo-ci’ ‘-fx’
Accurev command (0.592468): accurev ‘hist’ ‘-s’ ‘orbitz-api-foo-ci’ ‘-t’ ‘now.20′ ‘-fx’
Accurev command (1.195955): accurev ‘show’ ‘streams’ ‘-fix’ ‘-s’ ‘orbitz-api-foo-ci’ ‘-fx’
[snip]

Once that information is loaded, they render an Atom feed to summarize the transaction information. Items like transaction type, user, comments, and number of files are displayed. Additionally, building a page that summarizes the transaction lets them show file status like modified, added, or removed. A tiny bit more complicated, but again can be easily accomplished with AccuRev CLI. Here’s a snippet of that processing:


[snip]
Parameters: {“stream”=>”orbitz-api-foo-ci”, “action”=>”transaction”, “id”=>”4587″, “controller”=>”accurev”}
Accurev command (0.937849): accurev ‘show’ ‘streams’ ‘-fix’ ‘-s’ ‘orbitz-api-foo-ci’ ‘-fx’
Accurev command (0.356075): accurev ‘hist’ ‘-s’ ‘orbitz-api-foo-ci’ ‘-t’ ’4587.1′ ‘-fx’
Accurev command (0.603777): accurev ‘hist’ ‘-fx’ ‘-p’ ‘orbitz-api-foo’ ‘-k’ ‘defunct’ ‘/./build.xml’ ‘/./foo.txt’ ‘-fx’
Accurev command (0.333936): accurev ‘anc’ ‘-p’ ‘orbitz-api-foo’ ‘-v’ ‘orbitz-api-foo-ci/3′ ‘-1′ ‘/./foo.txt’ ‘-fx’
Accurev command (0.357128): accurev ‘anc’ ‘-p’ ‘orbitz-api-foo’ ‘-v’ ‘orbitz-api-foo-ci/44′ ‘-1′ ‘/./build.xml’ ‘-fx’
[snip]

Finally, that summary page provides the ability to view the file contents and run diffs. Here’s a sampling of how the unified diff gets generated:


[snip]
Parameters: {“eid”=>”56″, “action”=>”view_diff”, “previous_version”=>”721/45″, “controller”=>”accurev”, “virtual_version”=>”106/44″, “path”=>”/./build.xml”, “depot”=>”orbitz-api-foo”}
Accurev command (0.309664): accurev ‘cat’ ‘-p’ ‘orbitz-api-foo’ ‘-e’ ’56′ ‘-v’ ’106/44′
Accurev command (0.637650): accurev ‘cat’ ‘-p’ ‘orbitz-api-foo’ ‘-e’ ’56′ ‘-v’ ’721/45′
[snip]

Orbitz has built in a number of other optimizations and naturally this post only reveals a glance into what they’ve accomplished. But to give you an idea of the kind of volume this can support, there are anywhere from 9,000 – 10,000 requests a day for the RSS URL during the work week! Their users have given very positive feedback regarding the usefulness of this page and more enhancements are in the works, like links in from issue tracking systems.

So the next morning you come in to work, get your coffee, and fire up your Google Reader to see what’s new in the world you care about, and think about how cool it would be to automatically be able to check on all the latest code changes from your outsourced team overseas…

6 comments »

Posted in AccuRev, Tips and Tricks

Tags:

Developer Recipes: AccuRev + JIRA + Eclipse using Mylyn

April 9th, 2008 by admin
Share|

Related Recipes: AccuRev + Eclipse + Ruby.

In this developer-centric recipe I am going to setup a power-tool trifecta consisting of Eclipse, JIRA, and AccuRev. I’m not talking about installing each independently. No, no, no. AccuRev + JIRA + Eclipse ScreenshotThis recipe is going to take things one step further and configure full bidirectional integrations for a wickedly powerful, fully integrated development environmentAccuRev + JIRA + Eclipse Chart where the majority of common day-to-day development tasks can be done right within Eclipse (right picture). Integrations are the crux of setting up a best-of-breed tool strategy and if you use these three tools you definitely want them talking together (left picture). Enough chop, let’s rock.

Install Applications

Let’s start by covering the basics and installing the latest versions of all three tools.

  1. Install AccuRev 4.6.x. download. Follow the install wizard. See the quick setup guide to import code and setup streams [Windows page 1 / Linux-page 13].
  2. Install Eclipse 3.3.x. download. Follow the install wizard. See documentation if needed.
  3. Install JIRA 3.12.x. download. Follow these instructions.

At this point, you have three independent tools installed. Developers can checkin/checkout code from AccuRev, use Eclipse to modify the source code, and track bug/feature development with JIRA. In all, not a bad setup. But toggling between all these tools just takes valuable time away from coding and there is no mashing of logically related meta-data to generate useful reports… such as:

  • “Which files/lines fixed issue #1234?”
  • “Was bug #5678 fixed in mainline, 2.x, and 1.x codelines?”
  • “Release is this Friday. How many issues are unresolved in the QA area and who are we waiting on?”
  • “Which fixes went into Release 4.5.101?”
  • “If I start working on the 4.x codeline, which known fixes will I be compiling against?”

Integrate Eclipse + AccuRev

Let’s eliminate jumping between Eclipse and the AccuRev CLI or StreamBrowser GUI. Rather, why not just keep/promote/update/merge directly within Eclipse. You can install our native Eclipse plugin via the Eclipse software updater.

  1. Help –> Software Updates –> Find/Install
  2. Select ‘Seach for new features to install’
  3. Create ‘New Remote Site’ named ‘AccuRev’ with URL http://www.accurev.com/download/eclipseupdate/32
  4. Checkbox ‘AccuRev’ and select Finish

When you create a new Project, choose to “Checkout from AccuRev.” Now the Eclipse ‘Team’ menu has a sleu of AccuRev commands available for inline use and your file navigator has icons for version control status. Sweet. Note: there is an AccuRev quickstart PDF located in your Eclipse plugin directory (e.g. /opt/eclipse/plugins/com.accurev.eclipse_4.6.1.32/UsingAB4Eclipse.pdf).

Integrate Eclipse + JIRA

Have you heard of the Eclipse Mylyn project? Seriously, it’ll bring a tear to your eye. In short, Mylyn is a generic framework for ‘task management’ within Eclipse and has a number of connectors to tools like JIRA, Bugzilla, and Trac. Guess what? You can interact with JIRA directly within Eclipse. It’s sick! seriously. Once again, you can install directly from the Eclipse software updater. First install the Mylyn framework and then the JIRA connector.

  1. Install Mylyn 2.0. Follow this setup guide. Basically, just like the AccuRev plugin above, create a remote updater site with this URL: http://download.eclipse.org/tools/mylyn/update/e3.3
  2. Install JIRA Connector. Follow the short setup guide provided by Atlassian.

As a developer, your world just got a whole lot better. Not only can you commit/update file changes directly within AccuRev, now you can open/close/update JIRA issues all without leaving Eclipse. Nuts!

Integrate AccuRev + JIRA

The final piece to the puzzle. Wait? What does hooking AccuRev to JIRA actually mean?!

Lets take a step back. Back in the day, using one of those other traditional branch-based SCM tools, you probably entered your bug #id into the commit comment. Commit 10 files. Enter bug #1234. Commit another 7 files. Enter same bug #1234. Very loose. And you probably had some validation and reporting scripts all stacked on top to keep things (hobbled) together. At the end of the day, this was far from a ‘tight’ integration and a struggle to maintain and enforce. Answering the simple question “which files -and- lines were fixed for bug #1234″ was not easy (probably impossible!).

AccuRev Change Packages. AccuRev has an out-of-the-box feature called Change Packages that natively tracks a set of files (as patches) regardless of the number of commits. Change packages are first class citizens in AccuRev. You can promote multiple times to the same change package and even remove files. The trick is to understand that a change package has a 1-to-1 relationship with… an issue! And those issues can come from JIRA. So as you work in your AccuRev workspace coding day-to-day you can promote your changes and assign them to a JIRA issue. Then make more changes and promote to the same issue. Think of it like this: creating a feature or fix may take 1 day or 100 days; 1 commit or 50 commits; Change packages don’t care. They just track the current set of files that you claim are ‘logically’ related as part of a fix or feature. That’s it! I’ll keep this short, but basically, you can now use the change package / issue as a new, first-class currency for promoting further up and/or patching those changes to other codelines! It’s very powerful. See pg 33 of our Concepts Guide for details. But I digress.

Back to the recipe. Hooking up AccuRev and JIRA means that AccuRev receives issues from JIRA and JIRA receives meta-data such as fixed files and versions for each issue. The setup requires our own connector technology called AccuBridge for JIRA. It’s pretty easy to setup and simply requires mapping JIRA fields to AccuRev fields and setting up the data synchronization. There is a well written ‘quick start’ guide to walk you through the entire setup process.

  1. Install AccuBridge for JIRA. download (link half-way down). Follow the setup guide located in the download package: doc/ais4jira_quick.pdf. [Note: Additional licensing may be required]
  2. Enable Change Packages. See pg 75 of the Admin Guide. Basically, you need to tell AccuRev ‘when’ to prompt users for issues, ‘which’ issues to query for, and which data columns to display in the lists. [Note: AccuRev Enterprise Edition is required]

At this point, developers are promoting file changes to JIRA issues and JIRA can report on ‘who’ fixed ‘which’ files for any given bug/feature/task. Now that’s what I call traceability!

All Together Now!

With everything hooked up, what do we have? Simple. AccuRev + JIRA + Eclipse ScreenshotAs a developer you can do just about everything directly within Eclipse. Edit files. Commit and update changes from AccuRev. Create issues and update comments/status/fields in JIRA. And behind the scenes, the JIRA records are being annotated by AccuRev as files are promoted out of Eclipse. Eclipse is your one-stop-shop workbench for developing, tracking changes, and managing task workflow (see picture).

Next we’ll add a build tool to the mix that integrates with AccuRev, JIRA, and Eclipse… But that’s for another day. Sounds great though, doesn’t it!

/happy coding/ – dave

7 comments »

Posted in AccuRev, Best Practices

Tags: