Archive for the ‘continuous integration’ category

Agile Kids Say the Darndest Things

November 28th, 2011 by lorne cooper

I hope I don’t end up with a seized engine on the side of the road, but if I do, I’ll know I should have had that oil change. I hope I don’t end up on the Worst Dressed List, but if I do, at least I’ll know I should have given away those old shirts.  I feel sorry for those on the “Worst Agile Implementation” list who don’t even know they’re there.

In the past few months I’ve had the privilege of talking to approximately fifty organizations about their Agile implementation.  Most of them are doing well, and many of them have great insights about how they customized Agile to fit their process requirements.  But some of them really Say the Darndest Things.

“We do Scrum, it’s just the rest of the company doesn’t.”

“So first we break the requirements specification into pieces and call each of the pieces a story.  Then we do our iterations and pass them off to the release team.  We’d sure like to get Product Management, QA, and the customer involved, but they don’t want to.”

There are a lot of places an Agile approach can add value, and I’d hate to adopt a “waterfall approach to going Agile”, but you’re really not doing Scrum.  The biggest chunks of value, the incremental use of customer feedback, and going from “completed state” to “completed state” in each iteration are lost if you can’t get more support.

“We’re Agile until the development is done.”

More than once I’ve been speaking with an earnest development leader who’s describing the Scrum process.  They’ll launch in, with obvious pride, and tell me how they’ve gone to two week iterations, do standup meetings, and work from a backlog.  “Terrific!  And how do you do QA?”

Oh, yes, of course they do QA, silly!  In fact, they demo the completed development to the QA team every sprint review and send it off to get tested.  Sometimes, unfortunately, QA actually finds some bugs that need fixing.  So that’s why they put the sprint on hold for a while to fix the bugs and loop them back into QA “’cause we don’t want to wait an entire sprint before they can restart the testing.”

The other side of this one is the guys that take the old “Release Tail” loophole for all it’s worth.  “Yes, Lorne, we’ve been agile for three years now.  We do Scrum, unit testing, standups, and play in the World Series of ‘Planning Poker’.  We do that for about six weeks, or until the release.  Then we have a three month release testing tail, which follows a ‘modified Scrum process’ … the project leader estimates the amount of work on each bug QA finds, and assigns it to a developer.  Sure, sometimes we have to work on new functionality during the “release testing tail” … you can’t expect the customer to stop needing improvements for three months!”

Folks, I don’t think I’m sharing any great trade secret when I tell you the QA process needs to be completed before the story is considered “done.”  I don’t want to be Klaus Fuchs of Scrum, but here’s the secret: you’re going to have to invest more in testing up front.

“We do continuous integration every night.”

I blame the education system: how’s an engineer supposed to know what “Continuous” means when we have “social promotion!”  Now some people understand the idea of continuous integration, and made a conscious effort to make it more “Discrete”.  Some companies I talked to had broken builds that lasted for a week.  You’d rather have a child repeating “Mummy” every 30th of a second before you’d like to get an email every five minutes saying the “Build Failed.”  I get it.  And if the email was going to your boss too, well, you don’t have to be Dogbert to know that’s a bad idea.

Builds are going to fail.  Get used to it.  The problem is not that the build failed, but that you couldn’t fix it.  Good practices are to have the team drop what they’re doing when the build fails and hop on fixing it.  If they can’t fix it, it needs to get escalated *pronto*.  Better is to have the team do local builds and unit testing before they check in.  Best Practices are to divide up the build process by team and stage of development, so your team only pollutes itself, not the rest of the development org.

“We don’t need training since we can use the internet.”

Uh huh. So I guess the schools will be shutting down any day now.  Not that the Internet might not turn out to be a useful aid someday, but the software development process is a hands-on activity.  And similar to other hands on activities, like dancing or carpentry, you can’t learn to do it by reading a book.  You’re going to need to get some experience with the process before you understand how to run a sprint review or a stand up, how to estimate stories, and how to work with your QA partner.

Now if you’re a hobbyist and working for free, your time is cheap, and there’s no reason not to use trial and error as a learning method.  But if you’re getting paid, and your work is important, you really don’t want to waste four sprints figuring out what someone can help you get right in sprint two.

I’m hoping my surgeon, pilot, and barber got a few lessons before it was my turn.

Finally…

No one has to pass a test to call themselves “Agile,” nor should they. Agilistas don’t have a monopoly on the right way to develop software.  But when people believe they’ve made it to Agile without using critical Agile concepts like time boxing development or getting to “done”, they’re missing the real value.

 

What’s New in AccuRev Version 5.2?

August 22nd, 2011 by damonpoole

I’m very excited about our 5.2 release! We’ve completed the move to PostgreSQL on the back end, fully internationalized our products, and added a slew of new features, like per-element security so that you can lock down certain files or directories to specific groups or users.

In addition to moving to PostgreSQL, we’ve taken advantage of its capabilities to increase performance in a few key areas such as update, especially when using cross-links, populate, and more. And as a first step towards fully embracing 64bit servers, we now support 64bit CPUs on Linux servers. You can look forward to additional performance boosts as we exploit the capabilities of PostgreSQL in the future.

Haven’t upgraded in a while? Check out what else is new! To get a full flavor of all of the new functionality we’ve added recently, check out recent webinar which highlights what’s new.

If you haven’t tried our plug-ins in a while, we’ve put out new releases of our Eclipse and Visual Studio plug-ins for all versions of AccuRev. Eclipse and Visual Studio plug-ins will work with older releases, so no need to wait for 5.2 to start benefiting from the new features in the plug-ins.

We’ve fully integrated the Web UI into the plug-ins so you can now use all of the following functionality: version browser, annotate, stream browser, and all of AccuWork. Also, in 5.2 you can now access the Web UI with a single click from most screens. For instance, if you want to send somebody a URL of a file or diff, you can just click the “copy to clipboard” button in the toolbar and then compose an e-mail to somebody and paste in the URL. When they click the URL, they will go right to that file or diff. And the Web UI now allows you to print any table that it supports.

While we are in the process of making the new functionality in the Web UI available in all of our UIs, this is a great first step to quickly link to the new functionality.

A Few More Details on What’s Available Via the Web UI
If you haven’t used the Web UI or are using an older version, you may not have heard that there is an entirely new version browser in the Web UI and for AccuWork users- AccuWork on the Web has been completely revamped. For instance, it is easy to search by keyword and you can do drag and drop query editing.

5.3 and Beyond… Here We Come!
The database and internationalization are two very important infrastructure changes which will accelerate both our engineering velocity as well as our market penetration. With our upcoming 5.3 release we will be re-introducing our quarterly pace of releases as we had with our 4.8 and 4.9 releases. With the switch to PostgreSQL and an internationalized code base complete, you will see the release of new functionality faster than ever before. In preparation for 5.3 and beyond, we’ve just launched a new product survey as one of the many ways we collect product priorities. Make sure to look for it and fill it out. It is one of the best ways to have a significant impact on our product roadmap. If you haven’t gotten an e-mail yet, you may not be in our survey database. Just ask your AccuRev administrator for a link.

Best Practices to Optimize Continuous Integration

June 28th, 2011 by AccuRev

There are a handful of SCM best practices that can optimize continuous integration.  This post will look at:

  • Establishing a staging and isolation hierarchy
  • Automating builds at all stages in the hierarchy

Establishing a staging and isolation hierarchy for optimizing Continuous Integration

Proponents of continuous integration commonly suggest branching as little as possible and having developers work directly from the mainline as much as possible. However, this approach has several difficulties:

  • It puts the stability of the mainline at risk
  • It presupposes that traditional legacy branches are the only available isolation mechanism
  • It decreases the flexibility and agility required for fast iterative development

With modern SCM systems, a better approach is to implement a staging and isolation hierarchy for the development process. A staging and isolation hierarchy uses objects in the SCM system to represent the dependencies between development groups and process steps. For example, you may wish to model the following teams and activities:

  • Release engineering
  • Quality assurance
  • Product engineering
  • Component engineering

Each team or activity is assigned the equivalent of a private workspace (variously called “streams” or “branches” depending on the SCM system). Each team then receives the same benefits of private workspaces that individual developers receive.  With a staging hierarchy, changes move from less stable configurations to more stable as they are tested and deemed “good” for the next level. This allows the code to be stabilized as it gets ready for release without developer downtime. It also allows additional separation for each team if needed, so that the team’s changes can be integrated and tested before the components are integrated together.

Topaz Post 3 Best Practices to Optimize Continuous Integration

In this figure, there are four development teams as well as an area for accepting third-party code drops.  The teams are located in different geographical areas. The hierarchy represents the normal flow of changes through development from stage to stage. In the example of the above figure, changes provided by the GUI product engineering team in India flow from individual developer workspaces (not shown for brevity) to the GUI stage, where they can be continuously integrated and tested. Mature changes then flow to the UI_int stage and on to the QA and Release (Rel) stages, again being subject to continuous integration and testing at each stage. The web development team in Austin picks up well-tested changes from the UI_int stage and uses them as the basis of their development work; when the web changes are mature they can be pushed up the hierarchy and subject to broader testing in the UI_int, QA and Rel stages.

Using a development hierarchy provides more opportunities for check-pointing. Every change introduced into the system is a potential source of failure, and thus a potential checkpoint. If a change proves to be unstable, you can return both the source stage and the destination stage back to a previous checkpoint. By contrast, mainline development only offers you a single opportunity for check-pointing, specifically, the state of the main codeline itself. Unless your development process includes “freezing” the mainline for a long enough period to build, test and otherwise validate, the chances of isolating and check-pointing at an appropriately fine level of code granularity are slim, making any available checkpoints stale and of limited utility.

Automating builds at all stages in the hierarchy

In order to give developers prompt feedback about the changes submitted, the code must be built frequently, ideally several times per day. A continuous integration server such as CruiseControl, CruiseControl.NET or Draco.NET can be employed to automate this process. The continuous integration server periodically polls the SCM system for changes, populates the changes to the build server, initiates the build process, and reports the results of the build and unit tests.  It is important to note here that the continuous integration server utilizes the existing build scripts and build environment to execute the build. For example, if make is used to compile and link components written in C, then the continuous integration server will call the makefile to initiate the build process.  Because the continuous integration system uses the existing build, it is important for development groups to devote time and effort to:

  • Making the build as fast as possible,
  • Building automated unit tests and
  • Including unit tests as part of the build process.

Spending time on these items, even if it involves some rework of the build system to make it more compatible with a continuous integration environment, will improve not only the build process but the overall quality of the software release.

When utilizing continuous integration, it is crucial to communicate the results of the builds to the entire development team. Continuous integration system planners should consider a scalable communications method such as e-mail notification or an internal website to display build results.

Continuous integration servers such as CruiseControl come with built-in web reporting that can be easily customized, so that build results can be displayed on LCD panels in common areas at geographically dispersed locations. In this way, team members can easily see and respond to the build results and reduce the “fix latency” often encountered with nightly or weekly integration build approaches.