Posts Tagged ‘multistage continuous integration’

More Agile Methods and Practices Defined

August 30th, 2010 by damonpoole

In the last few posts I have discussed some of, what I consider to be, the most valuable Agile methods for development.  The list is pretty long, so breaking the list up allows me to define each practice and include the individual benefit of each Agile method.  This post defines some hot terms right now- continuous integration, multistage continuous integration, and story points.  Enjoy!

Agile Method: Continuous Integration

How frequently have you merged your code with changes from the mainline, only to find that the result doesn’t build, or it builds but it doesn’t work? Monthly? Weekly? Daily? Hourly? Or worse, how often have you made changes that broke the build, requiring you to quickly correct the problem while getting flames from your team members?

A practice that has emerged to address the problems of integration is called Continuous Integration. The basic idea is that if integrating changes together and getting build and test results on a regular basis is a good idea, integrating and getting build and test results on every change is even better.

With Continuous Integration, all work from all teams is integrated into a single codeline as frequently as possible. Every check-in automatically triggers a build and usually a subsequent run of the test suite. This provides instant feedback about problems to all interested parties and helps to keep the code base free of build and test failures. It also reduces the integration headaches just prior to release.

Agile Method: Multi-stage Continuous Integration

Integration is tough enough when you are just integrating your work with the work of other folks in your small team, or the whole effort is being done by a small team, but when you are part of a large team there is also something called “Big-Bang” integration. That’s the integration of the work that multiple teams have been working on for long periods of time. In a typical project, this integration is done in a phase toward the end of the project. During integration, many problems are discovered for the first time which leads to delays and/or changes in scope.

The real question is, what is a good way to structure this integration so that it will scale smoothly as you add more people to the equation? A good starting place is to look around for a pattern to follow. What are some similar situations? I have found that everything your organization needs to do in order to produce the best possible development organization can be entirely derived from the patterns and practices at the individual level. This approach makes it much easier to understand and much more likely that it will be successfully followed.

As individuals we work in transient isolation to reduce the impact of work in progress on each other. Organizations isolate WIP by using only official versions of 3pty sources and by producing official releases for customers.

Multi-stage continuous integration (MSCI) scales CI to large distributed environments by isolating work in progress at the team level. Changes move from individual to team to mainline as fast as CI allows, but stop on failure. MSCI is particularly important in a distributed environment where fixes to problems exposed by CI can be delayed by a full day

Agile Method: Using Story Points For Estimation, Instead of Units of Time

In my experience, the best unit to use for estimates is story points. Two different people with two different skill sets or levels of ability in an area may take different amounts of time to perform a particular task. Estimating in hours mixes together the scope of the work that needs to be done with the speed at which a particular individual can do that work.

On the other hand, story points are a relative measure of the scope of a user story. Story points separates out the “what” from the “who.” For instance, if you have one individual that is stronger with .Net than with Java, they will estimate a Java story as taking more hours than somebody that is stronger with Java. But they will probably both agree that something that is twice as easy to implement will take half as long to do.

To use story points, you need to create a relative scale of scope. A simple approach is to find a simple and straightforward story that you use to represent a single story point. Then think of stories that are 2, 3, 5, and 8 times larger in scope. You should have a couple of examples for each story point value to take into account that some stories have more test than coding, more documentation than test, etc.

Story points are primarily used for planning, not for implementation. Story points are used to help determine the contents of release by calculating a velocity.

Next up: Backlog, velocity, planning poker and burnup charts.

Yes, You Can! Doing Agile with Remote Teams

July 15th, 2010 by clucca

This past year I’ve attended several Agile conferences, presented at many of our own conferences, and traveled to Agile tradeshows sponsored by some influential industry-leading names. What surprises me most is the variance I see on the answer to this question: How do I do Agile with remote teams?

Some of the pure “Agileistas” will may answer this question in a manner that isn’t very possible for some of us in the real world, with “Don’t do it” or, “Reorg your company.”

I don’t’ know what those people expect here- is it possible that you can convince your management organization to tear down its office walls, move entire teams from across the country into one office space, just because you heard it at a conference that it was going to be really hard to do Agile remotely?

I certainly don’t believe that doing Agile with remote teams is a bad practice, nor do I believe that it’s impossible. Challenging? Yes it is. Easy to mess up? You bet. But there are some simple things you can do to avoid some of the pitfalls of remote organizations.

Agile with Remote Teams

Use face to face communication methods:

I just got the iPhone 4. It has face to face video chat. I also use Google Talk, and this also has video chat built in. It works great! With all the communication technologies we have now a days, there is no reason to avoid personal contact with remote teams.

If the remote team is a faceless organization, it will become a perceived impediment for the local team if there are problems. They wont’ be treated like part of the team, but more like an outside entity that drops code in and risks messing up the release.  We can bring these teams closer together to encourage communication, and allow them to adapt and respond to each other as issues arrive. Creating a persona and human link turns those faceless “code drops” into real people, people who you can reach out to. This gives the team the power to self manage your priorities, impediments and conflicts.

Create Agile ambassadors

We can even take face-to-face chat on the internet up a level. Sending ambassadors back and forth from the remote teams to home base and vice versa creates a human link that is deeper than any piece of technology canAgile for Remote Teams provide.  The ambassador’s job is to strengthen this link, because if the link is strong, each side will be more inclined to help each other.

Sometimes having a planning session with the remote team doesn’t give them the overall sense of how important the stories you’re working on might be. They may not feel as if it’s important, and that’s because they don’t know all the juicy details that led up to the creation of that story. Having an ambassador at that site gives that team visibility into all of the bits of information that make one user story important. In other words, the ambassador gives the entire back-story to an iteration (IE the gossip) so they can get a sense of how important something is, it’s not just a priority number in the ITS.

Use Tools That Work Globally

With all of the face-time, ambassadors, and communication, it’s essential that teams have a global view of what’s happening during the development cycle. It wouldn’t make much sense to reach out and then not provide a way to extend that communication on the development level.

Agile for Remote Teams

Imagine a team where having access to a user story or a piece of code wasn’t easy and available to them? This handcuffs the team tremendously.

Any remote team will need to be able to:

  • See each others user stories and tasks
  • Enter updates to user stories and tasks
  • Diff baselines and branches
  • Check out code from remote teams
  • Contribute to team discussions and wikis
  • Run continuous integration globally

Use Multi Stage Continuous Integration

Using multistage continuous integration lets people take a look at what’s been built, and if it functions correctly, give it to the other team. Having multi-stage set up gives you a way to integrate early and often, but only deliver changes that are “done”.

One of the main problems with remote development is integration, and it’s a double edge sword for most SCM tools. If you isolate the remote team too much, they won’t integrate often. And when they do integrate to mainline, they may break functionality. The problem with this is that they will not be able to respond to that change for 6-12 hours if your team is in another country. This basically means downtime for everyone.

But with multistage CI and AccuRev you can keep that team isolated and integrated at the same time.

Is it possible to do offshore Agile?

I’m not sure if it’s a question if it’s possible, I don’t think we have a choice. Offshore development is a reality that isn’t going away, and the simple answer of bringing teams together to practice Agile isn’t always variable.  Doing Agile with remote teams isn’t’ a choice, it’s a reality.

Continuous Integration vs. Continuous Perfection

March 26th, 2010 by clucca

Continuous integration is about confronting your risk. It’s about giving rapid feedback and responding to problems that you may encounter in your development environment.

If you’re Agile, you know one principle is adaptation. It’s not about sticking to the plan; it’s about exposing and responding to change in your development cycle.  This is why the principles of continuous integration and Agile walk hand in hand.

Delaying the inevitable risk of code integration doesn’t mean the risk will disappear.  Each line of code you change is a risk. Each commit you do is a risk. You can’t avoid the risk by putting it off.  Just like anything, if you were going to bottle up all of that risk, let it ferment for a few months and open it later… I’m pretty sure you wouldn’t want to drink the whole bottle in one gulp. It really only takes one sip to know the concoction is bad.

Continuous Perfection is one of the reasons we bottle up the risk. Let me give you an example.

We set up continuous build on trunk, we encourage our developers to integrate early and often, life is good…. until the day of the broken build.

If it’s broken for even a day, panic ensues, we’ve issued blame to developers for something they should not be responsible for, a stoppage in work for organization. People will feel rushed to resolve issues, the QA team will stop testing, and our release engineer will struggle to roll back code. Essentially, people will be running around like chickens with their heads cut off.

Our management teams do what they do best, they listen to the release team, QA team and the dev team. “Joe Developer broke the build, it’s his fault that we can’t work today”.

When did the broken build become such a big deal?

This scenario may repeat itself, over the course of several weeks our mangers will react, and start asking developers to put in process to avoid breaking the build. Our teams will start to feel that their code must be perfect before they commit. This problem is called Continuous Perfection, it’s the idea that the code must be perfect before it’s committed to the repository.

And doesn’t this seem to work against the principles of continuous integration? The idea here was that we integrate early and often to avoid Integration Hell and the Big Bang merge. We want to confront and work through our problems right now. But now we’ve started to go back to what we do naturally. Isolate changes, put off the inevitable and work off of our private branches. Our developers do local builds to make sure the builds work correctly, and jump through hoops to avoid committing to early in fear of being blamed. We are back where we started, with a huge merge at the end of the project that could fail everything.  We bottled up all our risk, and ended up in a more dangerous place then we intended.

But what can we do to solve this problem ?  To be continued…