Posts Tagged ‘continuous integration’

Continuous Integration: What’s Not to Love?

July 29th, 2010 by clucca

It seems like everyone loves continuous integration.  I’ll come out and say it- I love continuous integration! When we talk about the most widely adopted Agile practices, this one comes up the most.  Its positive benefits as a feedback mechanism provide a quantum leap forward in how development organizations think about their code.  I find it very difficult to see any downside in doing continuous integration, seriously what’s not to love?

Modern day continuous integration servers have 3 functions: Detect if a new build is needed, execute build, and notify people of the results. This is a great way of facilitating feedback to developers and allowing them to adapt and resolve problems.

But there’s something lurking in the shadows that nobody is talking about, maybe because people aren’t even aware that it’s a problem. Maybe it’s because we don’t want to ruin our love affair with CI and we’re all in denial.

What happens when those builds are done? What about the rest of “it”?

When I say “it” what I really mean is:

Most development environments include such things as complex application servers, automatic testing, release processes, compliance, audits, databases, 3rd party libraries, build dependencies, code analysis, unit tests… and another 1,000 other things I don’t have enough space to list here. How can you deal with this? Getting feedback to my dev team is a great targeted way to let people know code is broken, but isn’t this feedback useless if you can’t get the product out the door?

If I take an example build lifecycle of an application which is:

1.)    Build Application

2.)    Test

3.)    Deploy to environments (DB) (APP SERVER) <- by hand

4.)    Test

5.)    Redeploy (DB) (APP SERVER) (PROD) <-by hand

This may seem like easy in this example, but if we took all of these steps in the real world, this could represent hundreds of servers.  And this process will have to be repeated for every version of the product.

The crux of this issue is that if any of these steps are not performed 100% correctly, it translates to real dollar$ lost for the organization. These operations have to perform like clockwork.

Taking Continuous Integration to the Next Level

This is why I believe bringing continuous integration to the next level starts with the concept that the build produced from the CI server is just the beginning. Setting up a simple CI server and producing a build is easy, but managing it through the rest of its life is the real trick.

In a real example of this, we could take

1.)    Build

2.)    Run automated tests

  • If test succeeds: Deploy to to environments (DB) (APP SERVER)
  • If test fails: Notify dev team

3.)    Manual Testing in QA environment

4.)    Approval process

5.)    Redeploy (DB) (APP SERVER) (PROD)

Using AgileCycle RM

In this scenario I’m taking a version of the results from a CI build, run automated tests on them, monitor the test output and wait for success or failure. If there is success then deploy all components of the application, this includes a database components and a java war component. The build will then sit in QA for manual testing until its marked approved by the QA team, managers and operations team for deployment to production.

The idea here is if we automate these processes and decisions based on build, automated tests and approval process, you can produce code quickly and at a more rapid pace. If you can produce a clockwork-like automation around your build/test/deploy related processes, your team can spend time on what’s most important: Getting code out the door.

Yes, You Can! Doing Agile with Remote Teams

July 15th, 2010 by clucca

This past year I’ve attended several Agile conferences, presented at many of our own conferences, and traveled to Agile tradeshows sponsored by some influential industry-leading names. What surprises me most is the variance I see on the answer to this question: How do I do Agile with remote teams?

Some of the pure “Agileistas” will may answer this question in a manner that isn’t very possible for some of us in the real world, with “Don’t do it” or, “Reorg your company.”

I don’t’ know what those people expect here- is it possible that you can convince your management organization to tear down its office walls, move entire teams from across the country into one office space, just because you heard it at a conference that it was going to be really hard to do Agile remotely?

I certainly don’t believe that doing Agile with remote teams is a bad practice, nor do I believe that it’s impossible. Challenging? Yes it is. Easy to mess up? You bet. But there are some simple things you can do to avoid some of the pitfalls of remote organizations.

Agile with Remote Teams

Use face to face communication methods:

I just got the iPhone 4. It has face to face video chat. I also use Google Talk, and this also has video chat built in. It works great! With all the communication technologies we have now a days, there is no reason to avoid personal contact with remote teams.

If the remote team is a faceless organization, it will become a perceived impediment for the local team if there are problems. They wont’ be treated like part of the team, but more like an outside entity that drops code in and risks messing up the release.  We can bring these teams closer together to encourage communication, and allow them to adapt and respond to each other as issues arrive. Creating a persona and human link turns those faceless “code drops” into real people, people who you can reach out to. This gives the team the power to self manage your priorities, impediments and conflicts.

Create Agile ambassadors

We can even take face-to-face chat on the internet up a level. Sending ambassadors back and forth from the remote teams to home base and vice versa creates a human link that is deeper than any piece of technology canAgile for Remote Teams provide.  The ambassador’s job is to strengthen this link, because if the link is strong, each side will be more inclined to help each other.

Sometimes having a planning session with the remote team doesn’t give them the overall sense of how important the stories you’re working on might be. They may not feel as if it’s important, and that’s because they don’t know all the juicy details that led up to the creation of that story. Having an ambassador at that site gives that team visibility into all of the bits of information that make one user story important. In other words, the ambassador gives the entire back-story to an iteration (IE the gossip) so they can get a sense of how important something is, it’s not just a priority number in the ITS.

Use Tools That Work Globally

With all of the face-time, ambassadors, and communication, it’s essential that teams have a global view of what’s happening during the development cycle. It wouldn’t make much sense to reach out and then not provide a way to extend that communication on the development level.

Agile for Remote Teams

Imagine a team where having access to a user story or a piece of code wasn’t easy and available to them? This handcuffs the team tremendously.

Any remote team will need to be able to:

  • See each others user stories and tasks
  • Enter updates to user stories and tasks
  • Diff baselines and branches
  • Check out code from remote teams
  • Contribute to team discussions and wikis
  • Run continuous integration globally

Use Multi Stage Continuous Integration

Using multistage continuous integration lets people take a look at what’s been built, and if it functions correctly, give it to the other team. Having multi-stage set up gives you a way to integrate early and often, but only deliver changes that are “done”.

One of the main problems with remote development is integration, and it’s a double edge sword for most SCM tools. If you isolate the remote team too much, they won’t integrate often. And when they do integrate to mainline, they may break functionality. The problem with this is that they will not be able to respond to that change for 6-12 hours if your team is in another country. This basically means downtime for everyone.

But with multistage CI and AccuRev you can keep that team isolated and integrated at the same time.

Is it possible to do offshore Agile?

I’m not sure if it’s a question if it’s possible, I don’t think we have a choice. Offshore development is a reality that isn’t going away, and the simple answer of bringing teams together to practice Agile isn’t always variable.  Doing Agile with remote teams isn’t’ a choice, it’s a reality.

Continuous Integration- Like a Perfectly Functioning Toaster Oven

July 12th, 2010 by clucca

Where would we be in the world if we didn’t have feedback loops? Whether you know it or not, feedback loops are everywhere and you probably use them every day. Your toaster oven, your car, and the computer you’re using right now to read this blog post- feedback loops are everywhere.  Heck, even most biological organisms including humans and animals are constructed out of feedback loops. (Don’t believe me?) It’s the basis of all life!

A classic example of a feedback loop is the temperature control on your toaster oven. Toaster 300x300 Continuous Integration  Like a Perfectly Functioning Toaster Oven

Let’s imagine that we set the temperature to 300 degrees.

Here is the loop in which your toaster oven sets its temperature to 300 degrees:

1. Check thermostat for temperature

2. If greater than or equal to 300 degrees, stop heating coils

3. If less than 300 degrees, heat coils

4.  Goto step #1

“So what?” You’re probably saying to yourself, that’s easy.  Provide feedback to a device and it’s behavior will change. Why is this a big deal?

Continuous integration is the feedback loop for your development organization. It’s a feedback loop for people. It allows your developers to rapidly change course at a moment’s notice.

The Continuous Integration Feedback Loop

Let’s run through the continuous integration feedback loop, which is remarkably similar to the toaster oven.

Here is the feedback loop for a developer while using continuous integration:

1. Checkin code

2. Receive e-mail of build results

3. If build/test is success, enjoy the fruits of your labor

4.  If build/test is broken, fix and Goto step #1

If you don’t believe a feedback loop is necessary, if you’re thinking everything is working fine, you’re fooling yourself!  The reason? If there is a problem, you’re just not aware of it. In the same sense, would you just turn off the thermostat in your toaster?

Here’s how the toaster works without the feedback loop:

1. Heat coils

2.  Burn down house

And that’s what NOT doing continuous integration is like… playing with fire.

A lot of times I hear from customers who aren’t doing continuous integration that it’s just too hard to implement CI in their organization. The reasons vary, but usually come down to not being able to afford rewriting manual processes already in place in order to gain a rapid, consistent build running. To this I say, you are using the wrong tools, and bad practices to boot. The amount of rework and frustration you will save will be worth your effort. It’s a fools game not to expose your integration problems to the rest of the organization.

When I talk to customers who HAVE implemented a CI solution, I’m astonished at what I hear. They say “We never know we had so many issues until the QA cycle” or “We can’t’ believe we ever lived without this.”

Changing course is one of the key principles of Agile; inspecting and adapting is what allows your team to avoid disasters. It gives visibility into what’s happening with your current processes. Continuous integration is the tool that you need to allow your teams to function like a perfectly functioning toaster oven.

So what does a world look like without feedback? Houses on fire, cars crashing, development organizations failing and developers blindly making disastrous coding changes. Do yourself a favor, and think about feedback.