Posts Tagged ‘CI’

Pattern for Floating Labels with Streams

September 1st, 2009 by dave

I’ve encountered a number of use cases in the field where people enjoy the use of  ‘floating’ labels.  Implemented correctly, floating labels are a good example of convention-over-configuration  usually eliminating repetitive, manual build system updates.   Implemented incorrectly, they can be a traceability curse.   In this writeup I’ll show two examples of how to implement a floating label pattern in AccuRev.

Deprecated Concept? If you’re used to using floating labels as ‘markers’ on a branch [in another CM system] for stable code or feature complete state, you’ll find that the AccuRev stream hierarchy and associated workflow is a much better solution – in some cases, eliminating the need for floating labels altogether.  In my experience, floating labels have been used because the traditional CM system didn’t support process – so you had to roll your own – ultimately leading to lots of scripting and a solution that doesn’t scale.

Stories of Misuse. Every now and then I hear stories about how an urgent quick fix or missing doc change -needed- to be included in the “release-4.0.1″ that just got labeled in source control a few days prior. In some traditional CM systems, you have the ability to commit new file changes, un-label the old version and re-label the new version. As easy as this sounds, this anti-pattern instantly breaks traceability [do your labels maintain history?] and can quickly lead to confusion as “release 4.0.1″ has multiple meanings depending on when the label was pulled, either before or after the re-label. Example. I once asked someone why they routinely re-label individual files and their answer was – “because creating a new full label takes about 30 minutes.”  Why so long? Because most CM systems label the individual files (or references) in a given configuration; the more files you have, the longer it will take to label the entire configuration.  I’ve even heard stories about creating labels (and branches!) that take upwards of an hour each run.  Unless you are using a time/txn-based CM system, labeling build events is required if you need to track and compare changes between builds over time.  If you are waiting 30-min per label, forget about the agile “10-minute” build – you haven’t even gotten to the compile step yet!

Valid Cases. In practice there are  cases where having a floating label makes sense, typically related to your build system.  Most cases have to do with marking transient build configurations.  For example, when doing nightly or continual integration builds, let the build script or CI server simply pull/poll a single label and let the developers or build engineer determine what content is bound to the label. This way, the build server can be “dumb” and just get whatever the developer or build engineer say it should have.  This eliminates the need to configure your build system with each-and-every new build label – especially painful for those build systems requiring manual edits of config xml files – ugh.

The Pattern.   Two examples are presented below showing how floating labels can be used to support continuous integration with streams.

Static Floating Label.  In this example, continuous builds are performed on the single stream App_Integration.  No need to change your CI configuration. New builds are kicked off whenever changes are  promoted [by team leads, etc] from the child project streams.

Static Floating Label
Static Floating Label

The CI/build tool is configured to monitor the single App_Integration stream.  As the configuration changes, the name of the build stream remains unchanged – hence, the App_Integration stream is the ‘floating label’.

When doing very-frequent builds (e.g. per-promote or even nightly) this pattern works well as long as you have the discipline to immediately fix any broken configurations.

Dynamic Floating Label.

In this example, individual build configurations for App_Integration are marked with snapshots.  New snapshots are created whenever changes are promoted [by team leads, etc] from child project  streams, or on schedule (e.g. Nightly).

Dynamic Floating Label
Dynamic Floating Label

The stream Int_Good_Build is dynamically reparented to either the latest snapshot (e.g. latest build) or reparented to an alternate (e.g. prior) snapshot representing the last known good build.  The ability to move this stream, on-demand and as needed, represents a dynamic ‘floating label’ pattern.  Simply reparent the stream, update the attached workspace and the files on disk will mirror the current release snapshot configuration.  The same pattern can be applied to production releases where internal snapshots capture detailed per-configuration changes, but the reparented floating label can represent the latest public release.

Summary. In all, I’m a big fan of creating snapshots for each build, and removing old[er] ones along the way.  There are two camps on the subject, and this will be left for another blog.  Thus, I like the visibility of knowning which snapshots have been created (implying a purpose), and having a ‘marker’ stream as a floating label (e.g. context) to identify the current build being tested and/or delivered to an Int or Test machine.   Lastly, having recent snapshots immediately available assist with debugging why the latest build is broken – diff by snapshot.  sweet.

/Happy Coding/ dave

Continuous Integration: Methods of getting change

May 14th, 2008 by jsherwood

 Continuous Integration: Methods of getting changeDo you remember the last time you were excited to go somewhere? Were you like a kid saying, “Are we there yet? Are we there yet?”

More likely you were the one getting ‘them’ there. I’m sure it got pretty annoying to keep hearing everyone in the car moaning, wanting some sort of distraction until they got there. Maybe you even had DVD players or some other distraction so you didn’t have to hear the questions.

Now think about how you use Continuous Integration (CI) . Do you have polling of your software configuration management (SCM) tool setup? What about your other process tools, do they poll your SCM as well? Guess what, it’s the same difficult trip. You have tools burdening your working SCM (who slaves away everyday to provide for these other tools!). Every hour/minute/seconds some tool is asking your SCM tool “Are we there yet?”. Doesn’t make a lot of sense does it? Think about how many tools you have running daily. You might have multiple CI machines setup, multiple reporting machines, deployment machines all asking the same question over and over.

Quite a burden.

Now think about what your users are doing during this time. They are looking for the same distractions you might have given the kids. They are off emailing their buddies about the latest game, or watching AccuRev on Youtube (OK, maybe something even more enjoyable on YouTube). Doesn’t sound very productive, yet these automation tools were meant to do just that, increase productivity.

So what do you do? Well, a lot of tools allow you to flip the model. Push your information, don’t pull it.

If you have a large enough development group this won’t be enough. Maybe there really are code changes going into the system every minute. If you also increase the granularity of the information going to your CI and reporting tools, these tools can then decide the correct time to be a burden.

You can also reduce the frequency that you give out the same information. If you have several tools (or stages in a tool) depending on the same answer, they can get the answer from a secondary source that gets populated once.

You also want to be diligent about your monitoring. If you see a periodic load on your tools, justify the load. If it looks strange that Kevin is checking the history of a stream every minute, it probably is. If instead you saw CIMonitor as the user it would be more explicit. And it would be obvious that this should change.

And really, changes like these apply to any tool you are using. Do you really need updated reports everytime a bug is fixed? What about every day instead? If you reduce the burden on your tools to a ‘necessary’ level, then they can be further used to answer other questions.

Are we there yet?

Continuous Integration: The temptation of the Antipattern

April 30th, 2008 by jsherwood

You’ve started using Continuous Integration (CI), but it’s not quite meeting your needs. You’ve started out simply enough, getting a build to happen on Linux or some other common environment. You’ve gotten comfortable enough to start using it in more and more places.

You hear the whispers of potential. Maybe they come from coworkers, maybe you hear them in your head, maybe you’re hearing the antipattern. Yeah he’s talking to you, and he’s very tempting. Go ahead, add more tests, he says. Automated documentation, yeah that will work. Run some performance and benchmarking. Run some reports it can take it he says. Worse yet, the antipattern is sitting in the cube next to you. The antipattern doesn’t like to check in his code until it is ‘just right’, and then it is a big chunk of messy merged globby spaghetti code I wouldn’t eat and your product shouldn’t either.

The antipattern is tempting. Even the “good book” of CI talks about being careful of the antipattern, and then goes on to say go ahead and add more to your CI process. Very very tempting.

Beware of the temptation! Don’t fall into the hands of the antipattern. It only results in disappointment. There are people out there just waiting to show that CI isn’t successful. They are waiting for the first real failed build, delayed delivery, or time spent working on a tool instead of the product to kill the usefulness.

I should know, because I’ve given into temptation. We had a good CI setup going, our build processes were running smoother, broken builds were being caught before they became critical. And then I added benchmarking. Worked great. We were getting additional reports on leaks and bottlenecks. And we had time to attack them and slowly make them less and less of a problem. We were being lean. But then I heard the whisper. Run reports it said. Build some doc it said. So I tried. I wanted automated reports, one less thing to do manually right? But I wanted them with every CI build. Now the builds were taking 1/2 hour to an hour. Not so bad. Until code started breaking. You see, at the same time that I was tempted so were others. And it was easy. Large chunks of code were being added, destabilizing builds. Tests were failing over and over until people stopped listening to them. There was no buffer between changes and the rest of the developers. Yeah, we were able to clean up the mess using AccuRev, but the CI damage had been done. Benchmarking fell by the wayside while I tried to get reporting done. And the build took days to stabilize even while we scraped the code clean of the problem areas. And this was when the antipattern was able to leverage our misstep. Leaks got into the code. Not enough to destabilze, but enough that product performance could suffer. Instead of catching it early we caught it late, and the cost was that much higher when we fixed the problems.

That was over 2 years ago, but I’m here to repent. I had forsaken the usefulness and message of CI. The temptation was to strong for me then, but I and our product have recovered. Non-critical automation is no longer a dream to be shared with CI, but as a post operation run more infrequently. Stability and Performance of our product our paramount and CI is helping us keep them in line. I have exorcised the antipattern and remain vigilant.