Posts Tagged ‘private workspaces’

SCM Best Practices and Continuous Integration Go Hand-in-Hand

June 15th, 2011 by AccuRev

There’s no denying that this has certainly been the Agile decade for the software development industry.  It’s evident all around us in this tenth year since the Agile Manifesto was created. Most companies and development organizations today have implemented some form or aspect of Agile methodology into their software development processes. Whether you’re aiming for pure Agile or a mixed/hybrid approach, proven best practices in all phases of the software development lifecycle are crucial to success.

This is especially true in the case of continuous integration, one of the foundational aspects of the Agile methodology. The concept of continuous integration, as defined by Martin Fowler, is “a fully automated and reproducible build, including testing, that runs many times a day.  This allows each developer to integrate daily, thus reducing integration problems.”

With this approach, developers can work more closely in parallel while identify problems and debugging on the fly, accelerating the development process and improving the quality of the finished product.  The benefits of continuous integration are tremendous, but can quickly be eradicated if software configuration management (SCM) best practices are not carefully followed.

There are a handful of SCM best practices that can optimize continuous integration.   Let’s start with a quick look at the first two:

  • Using an SCM system to store and version all source code
  • Utilizing private developer workspaces

Best Practice: Using an SCM System to Store and Version all Source Code

Parallel development and distributed software teams can make tracking changes a daunting task, especially with the frequent changes that occur when using continuous integration methods.

For this reason, it is important to employ a software configuration management (SCM) system to strictly version changes to the code base. In addition to versioning source code, everything needed to build the system should be placed under version control, including the following:

  • Third-party libraries
  • Properties files
  • Database schema
  • Test scripts
  • Install scripts

All developers should have at least read-only access to all files needed for the build and should obtain all such files directly from the SCM system. This approach ensures that developers are working with the latest build environment, and is preferable to the common but error-prone practice of placing such files on a shared file server.

To effectively implement continuous integration, all development groups should work from the same central source code repository so that the latest changes from other developers are easily and immediately available.

Best Practice: Utilizing Private Developer Workspaces

In order to fully realize the benefits of continuous integration, software development organizations need to ensure that developers can remain productive regardless of the overall state and stability of the project source code. To achieve this, private workspaces that give developers full SCM capability should be used. Private workspaces enable developers to

  • work in isolation
  • revert to known “good” states when needed
  • checkpoint their changes
  • share only mature, well-tested code with other team members

The benefits of isolation are bidirectional—it protects developers from incoming changes, and protects the shared code configuration from incomplete or incorrect changes from any one developer. By creating private workspaces, developers receive all the benefits of SCM for their personal use, including the ability to revert to a previous state, viewing and tracking of changes between software configurations, and setting aside changes to begin work on a different task.

Once a new known good state is reached (for example, when a developer completes engineering and testing work on a feature), developers should checkpoint their work, typically by “checking in” or “keeping” the local changes in the SCM system. The checkpoint ensures that the developer’s work is safe on the SCM server and that the checkpoint can be revisited at any time. However, since the changes have not been shared, other developers and teams are not affected.

When a developer breaks isolation and decides to share a code change, he or she is essentially making an assertion that the change has reached a higher level of maturity. This, coupled with the use of local developer builds, helps to ensure that only mature, well-tested code is passed on to the rest of the development team, a primary benefit of continuous integration.

Use Case: Fixing the Broken Build

November 4th, 2008 by rmohr

by Rob Mohr, AccuRev

In one of many travels and customer visits, I came across a very cool way that AccuRev was helping to improve the way development teams do their work. To be more specific, this group was using Change Packages integrated with the JIRA Issue Tracking system to manage changes across their various product releases. They also used CruiseControl for continuous integration that would kick off nightly builds and notify the team with the results of the build.

From what they told me, the success of builds has significantly improved since they started using AccuRev because of the ability for the developers to work in their own private workspaces where they can integrate and unit test before promoting their changes for the rest of the team. Although broken builds are, for the most part, a thing of the past, they will still occur once in a while and need to be fixed ASAP.

Here is how they do it with AccuRev

The stream structure below is a simpler view of their overall software development process, but will be sufficient to show the use case.

Promoting to the Integration Stream

To start, the 4 developers below have made changes in their workspaces that will be promoted and associated to 4 different issues.

b1 Use Case: Fixing the Broken Build

As you can see below, the integration stream (EntSoft_Client_Int) is “aware” of which issues are active in the stream. These are the new “change packages” introduced in the stream to be included in the next nightly build.

2 show issues Use Case: Fixing the Broken Build

Build Fails in the Integration Stream

The next morning, the team is notified that last nights build failed via an email notification from CruiseControl. They have also scripted CruiseControl to automatically enable a time based stream called the “Temp_Fix_Build” stream and assign the appropriate transaction number to rollback the change packages from last night.

b31 Use Case: Fixing the Broken Build

Assign the Developer to Fix the Build

One of the developers creates a workspace on the Temp_Fix_Build and “change palettes” over each change package one at a time.  This gives them the ability to mix and match change packages together to determine which one of them is the problem.

b4 Use Case: Fixing the Broken Build

Problem Solved

After the culprit is fixed, the repaired change package(s) are promoted back into the integration stream for all to share.

b5 Use Case: Fixing the Broken Build

QCon’07 San Francisco – Agile Conference

November 9th, 2007 by dave

For those tracking the movement of luminaries such as Martin Fowler and Kent Beck or looking for scalability advice from the architects at companies like Orbitz, Ebay, and Linked-In… QCon ’07 in downtown San Francisco is the place to be!

The conference is packed with senior architects, software engineers, and open-source contributors galore — over 400 were rumoredDamon Poole / Cliff Utstein - AccuRev - QCon’07 to be in attendance. With speaker topics ranging from enterprise scalability to Agile practices, the audience was nothing short of being at the top of their game. Huddled together at the entrance to the conference rooms were a David Thomas - AccuRev - QCon’07number of vendors showing off new warez including AccuRev. Here’s a shot of Damon Poole & Cliff Utstein (top-right), Dave Thomas (left), and John Wall (bottom-right). The AccuRev booth had a John Wall - AccuRev - QCon'07constant flow of folks amazed at how the stream-based architecture brings a refreshing approach to managing software configurations and supporting agile practices. We also had some cameo appearances from existing customers like Authorize.Net and Orbitz.com.

Agile development methodologies is a major theme of the conference. For someone looking for advice on agile, just standing in the middle of the exhibit hall is all it takes — everyone is talking about best practices, success stories, and failed attempts.

We had a constant stream of people intrigued by our stream-based architecture and inherent support for agile practices. Here’s a list of common discussion points:

private workspaces: commit-early, commit-often

stream inheritance: merge-early, merge-often and sharing iterations early and automatically with parallel development efforts

issue tracking integration: assign, deliver and track development activity to stories/issues/features

continuous integration: integrations with cruise control, finalbuilder, electric-cloud, and others

refactoring: IDE integrations with eclipse, Intelli-J, Visual Studio support application-wide refactoring with version control

staged workflows: organize distributed teams and isolate integration areas from testing areas for explicit and repeatable access to known configurations.

snapshots: guaranteed reproducibility of labeled configurations for builds known to be ‘good’

reparenting: retarget active development to known good configurations for testing or stable development

If you didn’t have a chance to attend this years QCon, be sure to put next years event on your calendar!

/happy conferencing/ – dave